{blurb[service.protocol]}
- -Connection
-{problem}
- {/if} - {#if result} -- {result.detail} -
- {/if} - -diff --git a/README.md b/README.md index acb40640..ecb80bfe 100644 --- a/README.md +++ b/README.md @@ -4,7 +4,7 @@
A self-hosted music server that picks what to play next.
-Minstrel serves your music library to the web, to its own Android app, and to any Subsonic client. Shuffle, radio, likes and play history are worked out on the server, so every device gets the same mixes and the same history. It can also find, download and file the albums you want, through a search service and a download client you run yourself. It is for anyone who keeps their own music collection and wants it to sound like more than a random shuffle. +Minstrel serves your music library to the web, to its own Android app, and to any Subsonic client. Shuffle, radio, likes and play history are worked out on the server, so every device gets the same mixes and the same history. It is for anyone who keeps their own music collection and wants it to sound like more than a random shuffle.
@@ -49,7 +49,7 @@ Open `http://
@@ -64,11 +64,11 @@ The quick start sets everything that's required. These are the settings people m
| `MINSTREL_BRANDING_APP_NAME` | The name shown in the header, the browser tab and link previews. |
| `MINSTREL_LOG_LEVEL` | `debug`, `info`, `warn` or `error`. |
-Search services, download clients, ListenBrainz, email and recommendation tuning are set in the web UI and take effect without a restart. The full file-based surface is in [`config.example.yaml`](./config.example.yaml); to use it, mount your copy at `/etc/smartmusic/config.yaml`.
+ListenBrainz, email, metadata sources and recommendation tuning are set in the web UI and take effect without a restart. The full file-based surface is in [`config.example.yaml`](./config.example.yaml); to use it, mount your copy at `/etc/smartmusic/config.yaml`.
## Setup pitfalls
-- **The library mount has to be writable by uid 1000.** Minstrel files the albums it downloads into the library, and it deletes a file when an admin asks it to. It never moves, renames or retags files that were already there. Mount the library `:ro` to forbid writes: those actions then refuse and say why.
+- **The library mount has to be writable by uid 1000.** Minstrel deletes a file when an admin asks it to, and when the duplicate resolver removes a copy. It never moves, renames or retags your files. Mount the library `:ro` to forbid writes: those actions then refuse and say why.
- **`MINSTREL_LIBRARY_SCAN_PATHS` names the path inside the container** (`/music`), not the folder on your host.
- **On the internet, put it behind HTTPS.** Plain `http://` is fine on a network you trust. For a public server, bind the port to `127.0.0.1` and put a reverse proxy in front: see [docs/hosting.md](docs/hosting.md).
@@ -103,11 +103,6 @@ Every image carries a signed Android app, all with the same key, so a phone can
- Duplicates, missing files and suspect copies are found and listed under **Collection** for an admin to fix.
- Missing album art and MusicBrainz ids are filled in automatically.
-**Getting music**
-- A wanted list: requests from the household, artists you follow, and albums whose files went missing.
-- Minstrel searches the indexers you configure, hands the best release to your download client, checks what arrives against MusicBrainz, tags it, and files it. A better copy replaces a poor one.
-- Off until you add a search service and a download client.
-
**Clients and accounts**
- A full web app, built into the server.
- A native Android app that ships inside the server image, keeps playing what it has cached when the server is out of reach, and updates itself from your server.
@@ -116,13 +111,11 @@ Every image carries a signed Android app, all with the same key, so a phone can
## Scope and responsible use
-Minstrel indexes the files you point it at and streams them to your own clients. It ships no indexers, no trackers, no torrent, Usenet or NZB engine of its own, and no DRM circumvention.
+Minstrel indexes the files you point it at and streams them to your own clients. It does not find or download music, and it does no DRM circumvention.
-Getting music is optional and does nothing until you configure it. It searches only the search service (Prowlarr or a Torznab indexer) and the download client (qBittorrent) you add yourself.
+What you put in your library is your responsibility. Copyright law applies to your collection as it does to any other software that plays a file; please respect it, and the terms of any service you connect.
-What you put in your library, and the sources you configure, are your responsibility. Copyright law applies to your collection as it does to any other software that plays a file; please respect it, and the terms of any service you connect.
-
-Minstrel is not affiliated with or endorsed by ListenBrainz, MusicBrainz, Prowlarr, qBittorrent or Subsonic.
+Minstrel is not affiliated with or endorsed by ListenBrainz, MusicBrainz or Subsonic.
## Development
diff --git a/cmd/minstrel/main.go b/cmd/minstrel/main.go
index c77c7e84..cbd6c607 100644
--- a/cmd/minstrel/main.go
+++ b/cmd/minstrel/main.go
@@ -25,16 +25,12 @@ import (
"git.fabledsword.com/bvandeusen/minstrel/internal/db"
"git.fabledsword.com/bvandeusen/minstrel/internal/eventbus"
"git.fabledsword.com/bvandeusen/minstrel/internal/gc"
- "git.fabledsword.com/bvandeusen/minstrel/internal/importer"
"git.fabledsword.com/bvandeusen/minstrel/internal/library"
- "git.fabledsword.com/bvandeusen/minstrel/internal/lifecycle"
"git.fabledsword.com/bvandeusen/minstrel/internal/logging"
"git.fabledsword.com/bvandeusen/minstrel/internal/mailer"
"git.fabledsword.com/bvandeusen/minstrel/internal/notifications"
"git.fabledsword.com/bvandeusen/minstrel/internal/playlists"
- "git.fabledsword.com/bvandeusen/minstrel/internal/reacquisition"
"git.fabledsword.com/bvandeusen/minstrel/internal/recsettings"
- "git.fabledsword.com/bvandeusen/minstrel/internal/requests"
"git.fabledsword.com/bvandeusen/minstrel/internal/scrobble"
"git.fabledsword.com/bvandeusen/minstrel/internal/scrobble/listenbrainz"
"git.fabledsword.com/bvandeusen/minstrel/internal/secret"
@@ -43,7 +39,6 @@ import (
"git.fabledsword.com/bvandeusen/minstrel/internal/subsonic"
syncpkg "git.fabledsword.com/bvandeusen/minstrel/internal/sync"
"git.fabledsword.com/bvandeusen/minstrel/internal/tags"
- "git.fabledsword.com/bvandeusen/minstrel/internal/wanted"
)
func main() {
@@ -203,8 +198,8 @@ func run() error {
if acoustIDErr != nil {
logger.Warn("acoustid settings: using defaults (lookup off)", "err", acoustIDErr)
}
- // One AcoustID client, so the lookup worker and the import's check share
- // its pace (AcoustID allows three requests a second per key).
+ // One AcoustID client, so every lookup shares its pace (AcoustID allows
+ // three requests a second per key).
acoustIDClient := acoustid.New(acoustid.DefaultBaseURL)
acoustIDLookup := library.NewAcoustIDLookupWorker(pool, logger.With("component", "acoustid_lookup"),
acoustIDSettings, acoustIDClient)
@@ -355,44 +350,12 @@ func run() error {
// same pool and bus.
notifier := notifications.New(pool, bus, logger.With("component", "notifications"))
- // The request reconciler completes approved requests once the library
- // holds what they asked for.
- requestReconciler := requests.NewReconciler(pool, logger.With("component", "requests"), bus)
- requestReconciler.SetNotifier(notifier)
library.SetNotifier(notifier)
- go requestReconciler.Run(ctx)
- // Release catalogue: re-reads discographies older than a
- // week from MusicBrainz, a batch an hour, through the process-wide
- // MusicBrainz pacer it shares with tag enrichment.
+ // Release track lists from MusicBrainz, read on demand by the duplicate
+ // resolver through the process-wide MusicBrainz pacer it shares with tag
+ // enrichment.
cat := catalogue.New(pool, nil, logger.With("component", "catalogue"))
- go cat.Run(ctx)
-
- // The wanted list: reads followed artists' discographies
- // daily for new releases and marks wants held once the library has them.
- go wanted.New(pool, cat, logger.With("component", "wanted")).Run(ctx)
-
- // Search and grab: every ten minutes, searches the
- // configured indexers for wants due, paced by each source's
- // searches-per-hour, and hands the best release to the download client
- // when auto-grab is on.
- go wanted.NewSearcher(pool, cat, logger.With("component", "search")).Run(ctx)
-
- // Import with verification: every minute, finished
- // downloads are checked against their album (AcoustID, lengths, tags)
- // and filed into the library with MusicBrainz tags, or refused.
- var printer importer.Printer
- if acoustIDSettings != nil {
- printer = &importer.AcoustIDPrinter{Settings: acoustIDSettings, Client: acoustIDClient}
- }
- imp := importer.New(pool, scanner, cat, printer, scanner.Roots, logger.With("component", "import"))
- imp.DataDir = cfg.Storage.DataDir
- go imp.Run(ctx)
-
- // Download lifecycle: every ten minutes, a download that
- // failed or that nobody shares is retried, then given up and its album
- // searched again; an imported one seeds, then is removed.
- go lifecycle.New(pool, logger.With("component", "lifecycle")).Run(ctx)
// Duplicate resolver: classifies the sweep's groups and, with auto-resolve
// on, merges the copies that are safe to merge.
@@ -404,19 +367,6 @@ func run() error {
TrackLists: cat.RefreshReleaseGroup, Identify: acoustIDLookup.IdentifyRecordings,
}).Run(ctx)
- // Missing-file re-acquisition: wants back albums whose files have been
- // gone longer than the grace window, for the search schedule to find.
- // Hourly, because the grace window is measured in hours.
- //
- // A settings-load failure is logged, not fatal: NewSettingsService always
- // returns a usable service holding the defaults.
- reacqSettings, reacqErr := reacquisition.NewSettingsService(
- ctx, pool, logger.With("component", "reacquisition"))
- if reacqErr != nil {
- logger.Warn("reacquisition: using default settings", "err", reacqErr)
- }
- go reacquisition.NewSweeper(pool, reacqSettings, logger.With("component", "reacquisition")).Run(ctx)
-
// library_changes compactor: daily, deletes
// rows older than the configured retention so the change-log table
// doesn't grow unbounded. Clients that drop offline longer than
@@ -495,9 +445,6 @@ func run() error {
srv.FingerprintSettings = fpSettings
srv.LoudnessSettings = loudSettings
srv.AcoustIDLookup = acoustIDLookup
- // The sweeper above holds this same instance, so a save from the admin
- // card changes what it does on its next tick.
- srv.ReacqSettings = reacqSettings
// The server's own backup: the loop runs the nightly schedule, and the
// admin page starts, tests and configures the same instance.
backups := backup.NewService(pool, logger.With("component", "backup"), buildinfo.Version)
diff --git a/config.example.yaml b/config.example.yaml
index b5b7bf7c..cdf70ede 100644
--- a/config.example.yaml
+++ b/config.example.yaml
@@ -30,7 +30,7 @@ library:
branding:
app_name: "Minstrel" # MINSTREL_BRANDING_APP_NAME
- description: "Self-hosted music server with Subsonic compatibility, smart shuffle, and music acquisition." # MINSTREL_BRANDING_DESCRIPTION
+ description: "Self-hosted music server with Subsonic compatibility and smart shuffle." # MINSTREL_BRANDING_DESCRIPTION
subsonic:
# Accept the plaintext `p=` password on /rest/*. Leave off unless a client
diff --git a/docs/hosting.md b/docs/hosting.md
index 2e553d8a..92384518 100644
--- a/docs/hosting.md
+++ b/docs/hosting.md
@@ -86,7 +86,7 @@ changes on every restart until an account exists.
## Backups
Minstrel can back itself up once a night: accounts, likes, playlists,
-listening history, requests and settings, in one file. It is off until an
+listening history and settings, in one file. It is off until an
admin switches it on under **Admin › Backups**. Anything Minstrel can fetch
or work out again (artwork, fingerprints, MusicBrainz data) is left out, and
so are the keys and passwords Minstrel keeps for other services, which are
@@ -145,7 +145,7 @@ the place of what the scan found.
The library scan then finds each file, and the artwork, tags, fingerprints
and loudness are fetched or worked out again over the following hours. Keys
for other services (ListenBrainz, AcoustID, artwork and tag providers,
-email, download clients) and the backup's own destination are entered again
+email) and the backup's own destination are entered again
under **Admin**. An app that synced with the old server starts its sync over.
## Android app
diff --git a/docs/security.md b/docs/security.md
index a295f345..9904eca6 100644
--- a/docs/security.md
+++ b/docs/security.md
@@ -53,7 +53,6 @@ they aren't checked.
Some credentials can't be hashed, because Minstrel has to send or compare the
value itself:
-- the API keys and passwords for indexers, Prowlarr and download clients;
- the AcoustID key, and the cover-art and tag provider keys;
- the SMTP password;
- the secret key of the S3 bucket backups go to;
diff --git a/internal/acquire/acquire_test.go b/internal/acquire/acquire_test.go
deleted file mode 100644
index ac0ebf16..00000000
--- a/internal/acquire/acquire_test.go
+++ /dev/null
@@ -1,180 +0,0 @@
-package acquire
-
-import (
- "context"
- "errors"
- "net/http"
- "net/http/httptest"
- "path/filepath"
- "strings"
- "testing"
-
- "git.fabledsword.com/bvandeusen/minstrel/internal/db/dbq"
- "git.fabledsword.com/bvandeusen/minstrel/internal/downloader"
- "git.fabledsword.com/bvandeusen/minstrel/internal/exchange"
- "git.fabledsword.com/bvandeusen/minstrel/internal/indexer"
- "git.fabledsword.com/bvandeusen/minstrel/internal/prowlarr"
-)
-
-func validUpdate() Update {
- return Update{Label: "Prowlarr", BaseURL: "http://prowlarr:9696", TimeoutMs: 30000, LocalPath: "/downloads", SearchesPerHour: 120}
-}
-
-func TestUpdateValidate(t *testing.T) {
- if err := validUpdate().validate(); err != nil {
- t.Fatalf("a valid form: %v", err)
- }
- for name, tc := range map[string]struct {
- edit func(*Update)
- field string
- }{
- "no name": {func(u *Update) { u.Label = "" }, "label"},
- "switched on with no address": {func(u *Update) { u.Enabled, u.BaseURL = true, "" }, "base_url"},
- "an address with no scheme": {func(u *Update) { u.BaseURL = "prowlarr:9696" }, "base_url"},
- "a timeout too short": {func(u *Update) { u.TimeoutMs = 10 }, "timeout_ms"},
- "a relative downloads folder": {func(u *Update) { u.LocalPath = "downloads" }, "local_path"},
- "a relative client folder": {func(u *Update) { u.RemotePath = "data" }, "remote_path"},
- "too many searches": {func(u *Update) { u.SearchesPerHour = 4000 }, "searches_per_hour"},
- } {
- u := validUpdate()
- tc.edit(&u)
- var invalid *InvalidError
- if err := u.validate(); !errors.As(err, &invalid) || invalid.Field != tc.field {
- t.Errorf("%s: err = %v, want an InvalidError on %s", name, err, tc.field)
- }
- }
- // Switched off, an empty address is a row still being set up.
- u := validUpdate()
- u.BaseURL = ""
- if err := u.validate(); err != nil {
- t.Errorf("switched off with no address: %v", err)
- }
-}
-
-func TestSourceFor_BuildsTheClientTheProtocolNames(t *testing.T) {
- for protocol, check := range map[string]func(Source) bool{
- ProtocolProwlarr: func(s Source) bool { _, ok := s.Search.(*prowlarr.Client); return ok && s.Fetch != nil },
- ProtocolTorznab: func(s Source) bool { _, ok := s.Search.(*indexer.Torznab); return ok && s.Fetch != nil },
- // Exchange's results are all magnets: nothing to fetch through it.
- ProtocolExchange: func(s Source) bool { _, ok := s.Search.(*exchange.Client); return ok && s.Fetch == nil },
- } {
- src := SourceFor(dbq.AcquisitionService{Code: protocol, Protocol: protocol, SearchesPerHour: 7})
- if !check(src) || src.Code != protocol || src.PerHour != 7 {
- t.Errorf("%s: source %+v", protocol, src)
- }
- }
-}
-
-func TestJobs_AnExchangeDoesWhatItsSwitchesAllow(t *testing.T) {
- ex := dbq.AcquisitionService{Kind: KindDownload, Protocol: ProtocolExchange, ServesSearch: true}
- if !Searches(ex) || Fetches(ex) {
- t.Error("an exchange with only search on should search and not fetch")
- }
- // A qBittorrent's switches mean nothing: it fetches, whatever they say.
- qb := dbq.AcquisitionService{Kind: KindDownload, Protocol: ProtocolQBittorrent, ServesSearch: true}
- if Searches(qb) || !Fetches(qb) {
- t.Error("a qBittorrent fetches and never searches")
- }
-}
-
-func TestTest_NoAddressIsNothingToReach(t *testing.T) {
- r := Test(context.Background(), dbq.AcquisitionService{Protocol: ProtocolProwlarr, Kind: KindSearch})
- if r.OK || !strings.Contains(r.Detail, "No address") {
- t.Fatalf("result = %+v", r)
- }
-}
-
-func TestTest_SearchReportsResultsAndARefusedKey(t *testing.T) {
- srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
- if r.Header.Get("X-Api-Key") != "good" {
- w.WriteHeader(http.StatusUnauthorized)
- return
- }
- _, _ = w.Write([]byte(`[{"title": "Radiohead - OK Computer"}]`))
- }))
- defer srv.Close()
-
- row := dbq.AcquisitionService{Kind: KindSearch, Protocol: ProtocolProwlarr, BaseUrl: srv.URL, ApiKey: "good", TimeoutMs: 5000}
- if r := Test(context.Background(), row); !r.OK || !strings.Contains(r.Detail, "1 results") {
- t.Errorf("a working key: %+v", r)
- }
- row.ApiKey = "bad"
- if r := Test(context.Background(), row); r.OK || !strings.Contains(r.Detail, "refused") {
- t.Errorf("a refused key: %+v", r)
- }
-}
-
-// fakeExchange answers the three native APIs.
-func fakeExchange(t *testing.T) *httptest.Server {
- t.Helper()
- srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
- switch {
- case r.URL.Path == "/api/downloads":
- _, _ = w.Write([]byte(`{"enabled": true, "running": true, "inbound_conns": 3, "downloads": []}`))
- case r.URL.Path == "/api/search":
- _, _ = w.Write([]byte(`{"total": 0, "results": []}`))
- case strings.HasPrefix(r.URL.Path, "/api/torrents/"):
- w.WriteHeader(http.StatusNotFound)
- _, _ = w.Write([]byte(`{"error": "unknown torrent"}`))
- default:
- w.WriteHeader(http.StatusNotFound)
- }
- }))
- t.Cleanup(srv.Close)
- return srv
-}
-
-func TestTest_AnExchangeReportsEachJobSwitchedOn(t *testing.T) {
- srv := fakeExchange(t)
- row := dbq.AcquisitionService{
- Kind: KindDownload, Protocol: ProtocolExchange, BaseUrl: srv.URL, TimeoutMs: 5000,
- ServesDownloads: true, ServesSearch: true, ServesEnrichment: true, LocalPath: t.TempDir(),
- }
- r := Test(context.Background(), row)
- if !r.OK {
- t.Fatalf("all three jobs: %+v", r)
- }
- for _, job := range []string{"Downloads:", "Search:", "Release details:"} {
- if !strings.Contains(r.Detail, job) {
- t.Errorf("detail %q does not report %s", r.Detail, job)
- }
- }
- row.ServesDownloads, row.ServesEnrichment = false, false
- if r := Test(context.Background(), row); strings.Contains(r.Detail, "Downloads:") || !strings.Contains(r.Detail, "Search:") {
- t.Errorf("only search on: %+v", r)
- }
- row.ServesSearch = false
- if r := Test(context.Background(), row); r.OK || !strings.Contains(r.Detail, "switched off") {
- t.Errorf("every job off: %+v", r)
- }
-}
-
-func TestTest_ADownloadClientNeedsAFolderMinstrelCanSee(t *testing.T) {
- srv := fakeExchange(t)
- row := dbq.AcquisitionService{
- Kind: KindDownload, Protocol: ProtocolExchange, BaseUrl: srv.URL, TimeoutMs: 5000,
- ServesDownloads: true, Category: DefaultCategory,
- LocalPath: filepath.Join(t.TempDir(), "not-mounted"),
- }
- r := Test(context.Background(), row)
- if r.OK || !strings.Contains(r.Detail, "cannot see its downloads folder") {
- t.Fatalf("a missing folder: %+v", r)
- }
- row.LocalPath = t.TempDir()
- if r := Test(context.Background(), row); !r.OK || !strings.Contains(r.Detail, `"minstrel" category`) {
- t.Errorf("a visible, writable folder: %+v", r)
- }
-}
-
-func TestDownloadClient_WrapsThePathMapping(t *testing.T) {
- d, err := DownloadClient(dbq.AcquisitionService{Protocol: ProtocolQBittorrent, LocalPath: "/downloads", RemotePath: "/data"})
- if err != nil {
- t.Fatal(err)
- }
- if _, plain := d.(*downloader.Client); plain {
- t.Error("a row naming two folders should come back wrapped in its mapping")
- }
- if _, err := DownloadClient(dbq.AcquisitionService{Protocol: ProtocolProwlarr}); err == nil {
- t.Error("a search protocol is not a download client")
- }
-}
diff --git a/internal/acquire/build.go b/internal/acquire/build.go
deleted file mode 100644
index 15760b53..00000000
--- a/internal/acquire/build.go
+++ /dev/null
@@ -1,45 +0,0 @@
-package acquire
-
-import (
- "context"
- "fmt"
-
- "github.com/jackc/pgx/v5/pgxpool"
-
- "git.fabledsword.com/bvandeusen/minstrel/internal/db/dbq"
- "git.fabledsword.com/bvandeusen/minstrel/internal/release"
-)
-
-// Build makes a Service from the settings as they stand: every enabled
-// search source behind one paced Fanout, the first enabled download client,
-// and the quality profile. Built afresh for each pass and each request, so a
-// changed setting takes effect on the next one.
-func Build(ctx context.Context, pool *pgxpool.Pool) (*Service, dbq.AcquisitionSetting, error) {
- q := dbq.New(pool)
- settings, err := q.GetAcquisitionSettings(ctx)
- if err != nil {
- return nil, settings, fmt.Errorf("acquire: read settings: %w", err)
- }
- sources, err := Sources(ctx, pool)
- if err != nil {
- return nil, settings, fmt.Errorf("acquire: read search sources: %w", err)
- }
- clients, err := usableServices(ctx, q, JobDownload)
- if err != nil {
- return nil, settings, fmt.Errorf("acquire: read download clients: %w", err)
- }
- s := &Service{Profile: release.NewProfile(settings.AllowedQualities), Memory: NewMemory(pool)}
- fan := &Fanout{Sources: sources, Pace: SearchPace}
- if len(sources) > 0 {
- s.Search = fan
- }
- if len(clients) > 0 {
- row := clients[0]
- client, err := DownloadClient(row)
- if err != nil {
- return nil, settings, fmt.Errorf("acquire: download client %s: %w", row.Code, err)
- }
- s.Grabber = &Downloader{Client: client, Code: row.Code, Landing: row.LocalPath, Fetch: fan}
- }
- return s, settings, nil
-}
diff --git a/internal/acquire/fanout.go b/internal/acquire/fanout.go
deleted file mode 100644
index c4f607c8..00000000
--- a/internal/acquire/fanout.go
+++ /dev/null
@@ -1,164 +0,0 @@
-package acquire
-
-import (
- "context"
- "errors"
- "fmt"
- "strings"
- "sync"
-
- "git.fabledsword.com/bvandeusen/minstrel/internal/indexer"
-)
-
-// Searching every source at once. A source that is down narrows a search rather
-// than failing it: the results come back with the list of who did not answer.
-
-// SourceFailure is one source that could not be reached.
-type SourceFailure struct {
- Code string
- Label string
- Err error
-}
-
-// PartialError is a search that reached some sources and not others. It is
-// returned alongside the results, not instead of them.
-type PartialError struct {
- Failures []SourceFailure
- // Answered is how many sources did answer. Zero means the search failed.
- Answered int
-}
-
-func (e *PartialError) Error() string {
- if e == nil || len(e.Failures) == 0 {
- return "some sources did not answer"
- }
- parts := make([]string, 0, len(e.Failures))
- for _, f := range e.Failures {
- parts = append(parts, fmt.Sprintf("%s: %s", f.Label, indexer.FirstLine(f.Err.Error())))
- }
- return "some sources did not answer — " + strings.Join(parts, "; ")
-}
-
-// Unwrap lets errors.Is find indexer.ErrUnreachable through a partial failure.
-func (e *PartialError) Unwrap() []error {
- if e == nil {
- return nil
- }
- out := make([]error, 0, len(e.Failures))
- for _, f := range e.Failures {
- out = append(out, f.Err)
- }
- return out
-}
-
-// Fanout searches every source and merges what comes back. It is both the
-// Searcher and the Fetcher, routing a .torrent fetch to the source that
-// offered the result.
-type Fanout struct {
- // Sources in code order: the fold keeps a duplicate's first arrival, so
- // the order decides which source a grab is fetched through.
- Sources []Source
- // Pace spaces each source's searches. Nil paces nothing.
- Pace *Pacer
-}
-
-// Search asks every source concurrently. Each client bounds its own call by
-// its configured timeout, so there is no shared deadline here.
-func (f *Fanout) Search(ctx context.Context, query indexer.Query) ([]indexer.Result, error) {
- if f == nil || len(f.Sources) == 0 {
- return nil, indexer.ErrUnconfigured
- }
- found := make([][]indexer.Result, len(f.Sources))
- errs := make([]error, len(f.Sources))
- var wg sync.WaitGroup
- for i, src := range f.Sources {
- if src.Search == nil {
- continue
- }
- f.Pace.spend(src)
- wg.Add(1)
- go func(i int, src Source) {
- defer wg.Done()
- found[i], errs[i] = src.Search.Search(ctx, query)
- }(i, src)
- }
- wg.Wait()
-
- out := make([]indexer.Result, 0, 32)
- var failures []SourceFailure
- answered, unconfigured := 0, 0
- for i, src := range f.Sources {
- if src.Search == nil {
- continue
- }
- if errors.Is(errs[i], indexer.ErrUnconfigured) {
- unconfigured++
- continue
- }
- if rl, ok := indexer.AsRateLimited(errs[i]); ok {
- f.Pace.hold(src.Code, rl)
- }
- if errs[i] != nil {
- failures = append(failures, SourceFailure{Code: src.Code, Label: src.Label, Err: errs[i]})
- continue
- }
- answered++
- for j := range found[i] {
- if found[i][j].Source == "" {
- found[i][j].Source = src.Code
- }
- }
- out = append(out, found[i]...)
- }
- switch {
- case answered == 0 && len(failures) > 0:
- return nil, &PartialError{Failures: failures}
- case len(failures) > 0:
- return out, &PartialError{Failures: failures, Answered: answered}
- case answered == 0 && unconfigured > 0:
- return nil, indexer.ErrUnconfigured
- }
- return out, nil
-}
-
-// Download fetches a result's .torrent from the source that offered it: a
-// Prowlarr link carries Prowlarr's key and only Prowlarr can redeem it.
-func (f *Fanout) Download(ctx context.Context, r indexer.Result) ([]byte, error) {
- if f == nil {
- return nil, indexer.ErrUnconfigured
- }
- for _, src := range f.Sources {
- if src.Code != r.Source {
- continue
- }
- if src.Fetch == nil {
- return nil, fmt.Errorf("%w: %s cannot fetch torrent files", indexer.ErrUnconfigured, src.Label)
- }
- return src.Fetch.Download(ctx, r)
- }
- if r.Source == "" {
- return nil, fmt.Errorf("%w: this result does not say which source it came from", indexer.ErrUnconfigured)
- }
- return nil, fmt.Errorf("%w: %q is no longer a configured source", indexer.ErrUnconfigured, r.Source)
-}
-
-// Narrowing is one source that did not answer, as a page shows it.
-type Narrowing struct {
- Source string `json:"source"`
- Label string `json:"label"`
- Reason string `json:"reason"`
-}
-
-// narrowings reads a PartialError out of err, and whether any source
-// answered. Nil when err is not a partial failure.
-func narrowings(err error) ([]Narrowing, bool) {
- var partial *PartialError
- if !errors.As(err, &partial) || len(partial.Failures) == 0 {
- return nil, false
- }
- out := make([]Narrowing, 0, len(partial.Failures))
- for _, f := range partial.Failures {
- out = append(out, Narrowing{Source: f.Code, Label: f.Label, Reason: indexer.FirstLine(f.Err.Error())})
- }
- return out, partial.Answered > 0
-}
diff --git a/internal/acquire/grab.go b/internal/acquire/grab.go
deleted file mode 100644
index 001814bd..00000000
--- a/internal/acquire/grab.go
+++ /dev/null
@@ -1,338 +0,0 @@
-package acquire
-
-import (
- "context"
- "errors"
- "path"
- "strings"
- "sync"
- "time"
-
- "github.com/jackc/pgx/v5/pgtype"
-
- "git.fabledsword.com/bvandeusen/minstrel/internal/db/dbq"
- "git.fabledsword.com/bvandeusen/minstrel/internal/downloader"
- "git.fabledsword.com/bvandeusen/minstrel/internal/indexer"
- "git.fabledsword.com/bvandeusen/minstrel/internal/release"
-)
-
-// Taking a release. One live grab per want: a second is tie-broken against
-// the download already running.
-
-// Tag goes on every torrent Minstrel adds, so they can be told apart from
-// anything else in the client.
-const Tag = "minstrel"
-
-// MissingClient is Taken.Missing when no download client is set up.
-const MissingClient = "download_client"
-
-// nearlyDone is how far along a running download stays whatever would
-// replace it.
-const nearlyDone = 0.9
-
-// Downloader is the grab half of a Service.
-type Downloader struct {
- Client downloader.Driver
- // Code is the client's acquisition_services row.
- Code string
- // Landing is the client's downloads folder as Minstrel sees it; each
- // grab lands in a folder of its own under it.
- Landing string
- // Fetch gets a result's .torrent from the source that offered it.
- Fetch Fetcher
-}
-
-// Origin says who took a release.
-type Origin struct {
- By string // OriginSchedule or OriginOperator
- User pgtype.UUID
-}
-
-// Contest is how a grab fared against the download already running.
-type Contest struct {
- // Kept: the running one stays and this grab was not made. Replaced:
- // this grab took its place.
- Kept, Replaced bool
- Reason string
- Rival dbq.Grab
-}
-
-// Taken is what came of a grab.
-type Taken struct {
- Chosen *Candidate
- // Added is the only field that means a download started.
- Added bool
- SavePath string
- Grab dbq.Grab
- // Gone: the search no longer offers the release.
- Gone bool
- // Refused: not taken; the reason is on Chosen.
- Refused bool
- // Unconfigured: Missing names what is not set up ("" for search).
- Unconfigured bool
- Missing string
- Contest *Contest
- Error string
-}
-
-// Take searches again and grabs the candidate named by key: a person's
-// grab. Searching again keeps the indexer's download link (which can carry
-// its key) on the server, and finds a release gone since it was shown.
-// force takes what the verdict would not.
-func (s *Service) Take(ctx context.Context, want Want, key string, force bool, origin Origin) Taken {
- if s.Grabber == nil {
- return Taken{Unconfigured: true, Missing: MissingClient}
- }
- found := s.For(ctx, want)
- switch {
- case found.Unconfigured:
- return Taken{Unconfigured: true}
- case found.Error != "":
- return Taken{Error: found.Error}
- }
- chosen := pick(found.Candidates, key)
- if chosen == nil {
- return Taken{Gone: true}
- }
- return s.Grab(ctx, want, chosen, found.Candidates, force, origin)
-}
-
-// Grab hands a judged candidate to the download client. The schedule calls
-// it with the candidate it has just judged; Take calls it after searching
-// again.
-func (s *Service) Grab(ctx context.Context, want Want, chosen *Candidate, all []Candidate, force bool, origin Origin) Taken {
- out := Taken{Chosen: chosen}
- if s.Grabber == nil || s.Grabber.Client == nil {
- out.Unconfigured, out.Missing = true, MissingClient
- return out
- }
- if chosen.Verdict == Blocked || (chosen.Verdict != Grab && !force) {
- out.Refused = true
- return out
- }
-
- unlock := lockWant(want.ID)
- defer unlock()
- contest := s.contest(ctx, want, chosen, all, origin)
- if contest != nil && contest.Kept {
- out.Contest = contest
- return out
- }
-
- add := downloader.Add{Magnet: chosen.Source.Magnet, Tags: []string{Tag}}
- hash := chosen.InfoHash
- if add.Magnet == "" {
- if s.Grabber.Fetch == nil {
- out.Error = "This release is a .torrent link, and nothing is set up to fetch it."
- return out
- }
- body, err := s.Grabber.Fetch.Download(ctx, chosen.Source)
- if magnet, ok := indexer.AsMagnet(err); ok {
- add.Magnet, err = magnet, nil
- }
- if err != nil {
- out.Error = indexer.FirstLine(err.Error())
- return out
- }
- if add.Magnet == "" {
- add.Torrent = body
- if meta, err := release.ParseTorrent(body); err == nil {
- if hash == "" {
- hash = meta.InfoHash
- }
- if refused := s.judgeFiles(ctx, want, chosen, hash, meta.Files, force); refused {
- out.Refused = true
- return out
- }
- }
- }
- }
- if hash == "" && add.Magnet != "" {
- hash = indexer.MagnetHash(add.Magnet)
- }
- add.SavePath = landingFor(s.Grabber.Landing, hash, chosen.Name)
-
- err := s.Grabber.Client.AddTorrent(ctx, add)
- if errors.Is(err, downloader.ErrDuplicate) && s.heldElsewhere(ctx, hash) {
- why := "Already in the download client outside Minstrel's category, where Minstrel cannot follow it (" +
- strings.TrimPrefix(indexer.FirstLine(err.Error()), downloader.ErrDuplicate.Error()+": ") + ")"
- if force {
- out.Error = why + "."
- return out
- }
- set(chosen, Blocked, Hard, CodeBlocked, why+".")
- if s.Memory != nil && hash != "" {
- _ = s.Memory.Block(ctx, dbq.BlockReleaseParams{
- InfoHash: hash, Name: chosen.Name, Reason: BlockGone, Detail: why + ".", WantID: want.ID,
- ExpiresAt: pgtype.Timestamptz{Time: time.Now().Add(GoneBlock), Valid: true},
- })
- }
- out.Refused = true
- return out
- }
- if err != nil && !errors.Is(err, downloader.ErrDuplicate) {
- if errors.Is(err, downloader.ErrUnconfigured) {
- out.Unconfigured, out.Missing = true, MissingClient
- } else {
- out.Error = indexer.FirstLine(err.Error())
- }
- return out
- }
- out.Added, out.SavePath = true, add.SavePath
- if s.Memory == nil {
- return out
- }
-
- var replaced pgtype.UUID
- why := ""
- if contest != nil && contest.Replaced {
- out.Contest = contest
- replaced, why = contest.Rival.ID, "Replaced by "+chosen.Name+": "+contest.Reason+"."
- }
- g, err := s.Memory.Grabbed(ctx, dbq.InsertGrabParams{
- WantID: want.ID, InfoHash: hash, Name: chosen.Name, Key: Key(*chosen), Indexer: chosen.Indexer,
- Source: chosen.Source.Source, Client: s.Grabber.Code, SavePath: add.SavePath,
- Quality: string(chosen.Tier), Origin: origin.By, Forced: force, GrabbedBy: origin.User,
- }, replaced, why)
- if err != nil {
- out.Error = "The download started, but could not be recorded: " + indexer.FirstLine(err.Error())
- return out
- }
- out.Grab = g
- // Removed only after the new download was added and recorded, so a
- // failure never leaves the want with nothing running.
- if replaced.Valid && contest.Rival.InfoHash != "" {
- if err := s.Grabber.Client.Delete(ctx, true, contest.Rival.InfoHash); err != nil {
- contest.Reason += ". The download it replaced could not be removed from the client: " + indexer.FirstLine(err.Error())
- }
- }
- return out
-}
-
-// heldElsewhere reports whether a release the client says it already holds is
-// one it does not list for Minstrel: held under another category. Recorded
-// as a grab, it would read as gone half an hour later and be taken again,
-// every pass. A client that
-// cannot answer now is given the benefit of the doubt, as the add was.
-func (s *Service) heldElsewhere(ctx context.Context, hash string) bool {
- if hash == "" {
- return false
- }
- listed, err := s.Grabber.Client.Torrents(ctx, indexer.NormalizeHash(hash))
- return err == nil && len(listed) == 0
-}
-
-// judgeFiles judges a chosen release again on its .torrent's file list, and
-// sets it aside when the files say it is not the album, so no later search
-// offers it again. force still takes it.
-func (s *Service) judgeFiles(ctx context.Context, want Want, chosen *Candidate, hash string, files []release.File, force bool) bool {
- chosen.Files = files
- if chosen.FilesCount == 0 {
- chosen.FilesCount = len(files)
- }
- veto, reason, _ := release.JudgeFiles(want.Want, files)
- if veto == "" || force {
- return false
- }
- set(chosen, Rejected, Hard, veto, reason)
- if s.Memory != nil && hash != "" {
- _ = s.Memory.Block(ctx, dbq.BlockReleaseParams{
- InfoHash: hash, Name: chosen.Name, Reason: BlockFiles, Detail: reason, WantID: want.ID,
- })
- }
- return true
-}
-
-var wantLocks sync.Map // want id → *sync.Mutex
-
-// lockWant settles two grabs for one want one after the other. The
-// one-live-grab index is the backstop for anything outside this process.
-func lockWant(id pgtype.UUID) func() {
- if !id.Valid {
- return func() {}
- }
- m, _ := wantLocks.LoadOrStore(id.Bytes, &sync.Mutex{})
- mu := m.(*sync.Mutex)
- mu.Lock()
- return mu.Unlock
-}
-
-// contest settles a grab against the download running for its want: a
-// person's grab beats the schedule's; between two of a kind the better
-// candidate wins; a tie, a download nearly done, or one that cannot be
-// removed stays. Nil when nothing is running.
-func (s *Service) contest(ctx context.Context, want Want, chosen *Candidate, all []Candidate, origin Origin) *Contest {
- if s.Memory == nil || !want.ID.Valid {
- return nil
- }
- running, ok, err := s.Memory.LiveGrab(ctx, want.ID)
- if err != nil || !ok {
- return nil
- }
- c := &Contest{Rival: running}
- if running.InfoHash != "" && running.InfoHash == chosen.InfoHash {
- c.Kept, c.Reason = true, "That release is already downloading for this album."
- return c
- }
- switch {
- case origin.By == OriginOperator && running.Origin == OriginSchedule:
- c.Reason = "you chose it over the one the schedule started"
- case origin.By == OriginSchedule && running.Origin == OriginOperator:
- c.Kept, c.Reason = true, "A download you started for this album is already running."
- return c
- default:
- var theirs *Candidate
- for i := range all {
- if (running.InfoHash != "" && all[i].InfoHash == running.InfoHash) || Key(all[i]) == running.Key {
- theirs = &all[i]
- break
- }
- }
- if theirs == nil || !better(*chosen, *theirs) {
- c.Kept, c.Reason = true, "A download for this album is already running, and this release is no better."
- return c
- }
- c.Reason = "it ranks above the one that was running"
- }
- if running.InfoHash == "" {
- c.Kept, c.Reason = true, "A download for this album is already running, and it cannot be identified to replace it."
- return c
- }
- if ts, err := s.Grabber.Client.Torrents(ctx, running.InfoHash); err == nil && len(ts) > 0 && ts[0].Progress >= nearlyDone {
- c.Kept, c.Reason = true, "A download for this album is already running and nearly done."
- return c
- }
- c.Replaced = true
- return c
-}
-
-// landingFor is the folder a download lands in: one per torrent, named by
-// its infohash, so two grabs never unpack into the same place.
-func landingFor(root, hash, name string) string {
- dir := hash
- if dir == "" {
- dir = slug(name)
- }
- if dir == "" {
- dir = "unnamed"
- }
- return path.Join(root, dir)
-}
-
-// slug keeps a folder name to characters every filesystem in the chain
-// takes.
-func slug(s string) string {
- var b strings.Builder
- for _, r := range strings.ToLower(strings.TrimSpace(s)) {
- if b.Len() >= 80 {
- break
- }
- switch {
- case r >= 'a' && r <= 'z', r >= '0' && r <= '9':
- b.WriteRune(r)
- case r == '-', r == '_', r == ' ', r == '.':
- b.WriteRune('-')
- }
- }
- return strings.Trim(b.String(), "-")
-}
diff --git a/internal/acquire/memory.go b/internal/acquire/memory.go
deleted file mode 100644
index 8e553e9f..00000000
--- a/internal/acquire/memory.go
+++ /dev/null
@@ -1,199 +0,0 @@
-package acquire
-
-import (
- "context"
- "encoding/json"
- "errors"
- "fmt"
- "time"
-
- "github.com/jackc/pgx/v5"
- "github.com/jackc/pgx/v5/pgtype"
- "github.com/jackc/pgx/v5/pgxpool"
-
- "git.fabledsword.com/bvandeusen/minstrel/internal/db/dbq"
- "git.fabledsword.com/bvandeusen/minstrel/internal/indexer"
- "git.fabledsword.com/bvandeusen/minstrel/internal/release"
-)
-
-// Block reasons, matching release_blocks_reason_check.
-const (
- BlockOperator = "operator"
- BlockFiles = "files"
- // BlockImport: the import found its files were not the album.
- BlockImport = "import"
- // BlockStalled: given up on because it would not finish; held
- // back until the block's expires_at.
- BlockStalled = "stalled"
- // BlockGone: its download client would not let Minstrel follow it, held
- // outside Minstrel's category or leaving the client twice; held
- // back for GoneBlock.
- BlockGone = "gone"
-)
-
-// GoneBlock is how long a release its client lost track of is held back:
-// long enough that a release which keeps vanishing is tried once a day rather
-// than on every pass, short enough that it is taken once the client lets go.
-const GoneBlock = 24 * time.Hour
-
-// Grab origins, matching grabs_origin_check.
-const (
- OriginSchedule = "schedule"
- OriginOperator = "operator"
-)
-
-// Memory is what searches and grabs leave behind: every candidate of a
-// want's last search, the grabs, and the releases set aside.
-type Memory struct {
- pool *pgxpool.Pool
-}
-
-// NewMemory returns a Memory over pool.
-func NewMemory(pool *pgxpool.Pool) *Memory { return &Memory{pool: pool} }
-
-// Keep replaces a want's candidates with this search's, in order.
-func (m *Memory) Keep(ctx context.Context, wantID pgtype.UUID, cands []Candidate) error {
- return pgx.BeginFunc(ctx, m.pool, func(tx pgx.Tx) error {
- q := dbq.New(tx)
- if err := q.DeleteWantCandidates(ctx, wantID); err != nil {
- return err
- }
- for i, c := range cands {
- reasons, err := json.Marshal(c.Reasons)
- if err != nil {
- return err
- }
- if c.Reasons == nil {
- reasons = []byte("[]")
- }
- folded := c.FoldedFrom
- if folded == nil {
- folded = []string{}
- }
- if err := q.InsertWantCandidate(ctx, dbq.InsertWantCandidateParams{
- WantID: wantID, Key: Key(c), Name: c.Name, InfoHash: c.InfoHash, Indexer: c.Indexer,
- Source: c.Source.Source, SizeBytes: c.SizeBytes, Seeders: int32(c.Seeders),
- FilesCount: int32(c.FilesCount), Format: release.Describe(c.Release), Quality: string(c.Tier),
- Year: int32(c.Year), Verdict: string(c.Verdict), Reason: c.Reason, Reasons: reasons,
- FoldedFrom: folded, Rank: int32(i),
- }); err != nil {
- return err
- }
- }
- return nil
- })
-}
-
-// blocked is every release set aside, by infohash, with the sentence a
-// candidate shows. A failed read blocks nothing: offering a release set
-// aside costs one grab a person can undo, while blocking everything on a
-// database blip would hide every result.
-func (m *Memory) blocked(ctx context.Context) map[string]string {
- if m == nil {
- return nil
- }
- rows, err := dbq.New(m.pool).ListReleaseBlocks(ctx)
- if err != nil {
- return nil
- }
- out := make(map[string]string, len(rows))
- for _, r := range rows {
- out[indexer.NormalizeHash(r.InfoHash)] = BlockSentence(r.Reason, r.Detail)
- }
- return out
-}
-
-// BlockSentence is what a blocked candidate says.
-func BlockSentence(reason, detail string) string {
- switch reason {
- case BlockFiles:
- if detail != "" {
- return "Set aside for what its files hold: " + detail
- }
- return "Set aside for what its files hold."
- case BlockImport:
- if detail != "" {
- return "Refused on import: " + detail
- }
- return "Refused on import: its files were not the album."
- case BlockGone:
- if detail != "" {
- return "Held back for now: " + detail
- }
- return "Held back for now: its download client lost track of it."
- case BlockStalled:
- if detail != "" {
- return "Given up on for now: " + detail
- }
- return "Given up on for now: it would not finish."
- }
- return "You set this release aside."
-}
-
-// Block sets a release aside and marks every kept candidate of it blocked.
-func (m *Memory) Block(ctx context.Context, p dbq.BlockReleaseParams) error {
- p.InfoHash = indexer.NormalizeHash(p.InfoHash)
- if p.InfoHash == "" {
- return errors.New("a release without an infohash cannot be set aside")
- }
- return pgx.BeginFunc(ctx, m.pool, func(tx pgx.Tx) error {
- q := dbq.New(tx)
- if err := q.BlockRelease(ctx, p); err != nil {
- return err
- }
- return q.MarkCandidatesBlocked(ctx, dbq.MarkCandidatesBlockedParams{
- InfoHash: p.InfoHash, Reason: BlockSentence(p.Reason, p.Detail),
- })
- })
-}
-
-// Unblock lets a release be taken again. Its kept candidates wait for a
-// person until the next search judges them afresh.
-func (m *Memory) Unblock(ctx context.Context, hash string) (bool, error) {
- hash = indexer.NormalizeHash(hash)
- var n int64
- err := pgx.BeginFunc(ctx, m.pool, func(tx pgx.Tx) error {
- q := dbq.New(tx)
- var err error
- if n, err = q.UnblockRelease(ctx, hash); err != nil {
- return err
- }
- return q.MarkCandidatesUnblocked(ctx, dbq.MarkCandidatesUnblockedParams{
- InfoHash: hash, Reason: "Allowed again. The next search judges it afresh.",
- })
- })
- return n > 0, err
-}
-
-// LiveGrab is the download running for a want, if there is one.
-func (m *Memory) LiveGrab(ctx context.Context, wantID pgtype.UUID) (dbq.Grab, bool, error) {
- g, err := dbq.New(m.pool).GetLiveGrab(ctx, wantID)
- if errors.Is(err, pgx.ErrNoRows) {
- return g, false, nil
- }
- if err != nil {
- return g, false, err
- }
- return g, true, nil
-}
-
-// Grabbed records a download started, ending the grab it replaced first so
-// the one-live-grab index holds.
-func (m *Memory) Grabbed(ctx context.Context, p dbq.InsertGrabParams, replaced pgtype.UUID, why string) (dbq.Grab, error) {
- var g dbq.Grab
- err := pgx.BeginFunc(ctx, m.pool, func(tx pgx.Tx) error {
- q := dbq.New(tx)
- if replaced.Valid {
- if err := q.EndGrab(ctx, dbq.EndGrabParams{ID: replaced, EndReason: "superseded", EndDetail: why}); err != nil {
- return err
- }
- }
- var err error
- g, err = q.InsertGrab(ctx, p)
- return err
- })
- if err != nil {
- return g, fmt.Errorf("record grab: %w", err)
- }
- return g, nil
-}
diff --git a/internal/acquire/pace.go b/internal/acquire/pace.go
deleted file mode 100644
index fb2352e3..00000000
--- a/internal/acquire/pace.go
+++ /dev/null
@@ -1,145 +0,0 @@
-package acquire
-
-import (
- "context"
- "sync"
- "time"
-
- "git.fabledsword.com/bvandeusen/minstrel/internal/indexer"
-)
-
-// SearchPace is the one Pacer every search shares. Process-wide because a
-// Fanout is built fresh for each search; a Pacer per Fanout would start empty
-// and pace nothing.
-var SearchPace = &Pacer{}
-
-// holdWithoutRetryAfter is how long a source that answered 429 without saying
-// for how long is left alone: one schedule tick.
-const holdWithoutRetryAfter = 10 * time.Minute
-
-// Pacer spaces the searches sent to each source by its searches_per_hour: a
-// spacing, not a burst bucket, because Prowlarr answers 429 to a few requests
-// in quick succession. Every search spends, the schedule's and a
-// person's alike; only the schedule waits.
-type Pacer struct {
- // Now and Sleep are the clock, replaceable in tests.
- Now func() time.Time
- Sleep func(context.Context, time.Duration) error
-
- mu sync.Mutex
- next map[string]time.Time
-}
-
-func (p *Pacer) now() time.Time {
- if p.Now != nil {
- return p.Now()
- }
- return time.Now()
-}
-
-func (p *Pacer) sleep(ctx context.Context, d time.Duration) error {
- if p.Sleep != nil {
- return p.Sleep(ctx, d)
- }
- t := time.NewTimer(d)
- defer t.Stop()
- select {
- case <-t.C:
- return nil
- case <-ctx.Done():
- return ctx.Err()
- }
-}
-
-// gap is the spacing a source's number asks for; zero is unpaced.
-func gap(perHour int) time.Duration {
- if perHour <= 0 {
- return 0
- }
- return time.Hour / time.Duration(perHour)
-}
-
-// readyAt is when every source may next be asked.
-func (p *Pacer) readyAt(sources []Source) time.Time {
- p.mu.Lock()
- defer p.mu.Unlock()
- var at time.Time
- for _, s := range sources {
- if s.Search == nil {
- continue
- }
- if n := p.next[s.Code]; n.After(at) {
- at = n
- }
- }
- return at
-}
-
-// Wait holds until every source may be asked, and answers false, without
-// waiting, when that is after the deadline or ctx ends first. All of them,
-// not any: a search while one is held back would be narrowed, and the want
-// recorded as looked for.
-func (p *Pacer) Wait(ctx context.Context, sources []Source, deadline time.Time) bool {
- if p == nil {
- return true
- }
- at := p.readyAt(sources)
- if !deadline.IsZero() && at.After(deadline) {
- return false
- }
- if d := at.Sub(p.now()); d > 0 {
- return p.sleep(ctx, d) == nil
- }
- return ctx.Err() == nil
-}
-
-// spend takes one search's slot from a source.
-func (p *Pacer) spend(s Source) {
- g := gap(s.PerHour)
- if p == nil || g == 0 {
- return
- }
- p.mu.Lock()
- defer p.mu.Unlock()
- if p.next == nil {
- p.next = map[string]time.Time{}
- }
- at := p.next[s.Code]
- if t := p.now(); at.Before(t) {
- at = t
- }
- p.next[s.Code] = at.Add(g)
-}
-
-// hold leaves a source alone after it answered 429, for as long as it asked,
-// whatever its searches_per_hour says.
-func (p *Pacer) hold(code string, rl *indexer.RateLimited) {
- if p == nil {
- return
- }
- d := rl.RetryAfter
- if d <= 0 {
- d = holdWithoutRetryAfter
- }
- p.mu.Lock()
- defer p.mu.Unlock()
- if p.next == nil {
- p.next = map[string]time.Time{}
- }
- if until := p.now().Add(d); until.After(p.next[code]) {
- p.next[code] = until
- }
-}
-
-// Ready holds until every source this service searches may be asked again,
-// and answers false when that is past the deadline. The schedule calls it
-// before each want; a person's search never does.
-func (s *Service) Ready(ctx context.Context, deadline time.Time) bool {
- if s == nil {
- return ctx.Err() == nil
- }
- if f, ok := s.Search.(*Fanout); ok && f != nil {
- return f.Pace.Wait(ctx, f.Sources, deadline)
- }
- return ctx.Err() == nil
-}
diff --git a/internal/acquire/search.go b/internal/acquire/search.go
deleted file mode 100644
index 73960b3a..00000000
--- a/internal/acquire/search.go
+++ /dev/null
@@ -1,408 +0,0 @@
-package acquire
-
-import (
- "context"
- "errors"
- "sort"
- "strings"
-
- "github.com/jackc/pgx/v5/pgtype"
-
- "git.fabledsword.com/bvandeusen/minstrel/internal/indexer"
- "git.fabledsword.com/bvandeusen/minstrel/internal/release"
-)
-
-// One search for one want, and what could be done about each result.
-// internal/release
-// says whether a name is the album and what quality it is; this package
-// asks in the right order and says why in words a person can act on.
-
-// Verdict is what can be done about one result.
-type Verdict string
-
-const (
- // Grab: the album, in a quality the profile takes.
- Grab Verdict = "grab"
- // Review: maybe the album; a person decides.
- Review Verdict = "review"
- // Refused: the album, in a quality the profile does not take. The only
- // verdict that names a setting.
- Refused Verdict = "refused"
- // Rejected: not the album.
- Rejected Verdict = "rejected"
- // Blocked: a release set aside, by a person or by its files.
- Blocked Verdict = "blocked"
-)
-
-// order sorts verdicts: what can be taken first, what was given up on last.
-var order = map[Verdict]int{Grab: 0, Review: 1, Refused: 2, Rejected: 3, Blocked: 4}
-
-// Reason kinds: a hard reason is a fact about the bytes, a soft one about the
-// moment or a setting.
-const (
- Hard = "hard"
- Soft = "soft"
-)
-
-// Reason codes beyond internal/release's vetoes.
-const (
- CodeBlocked = "blocked"
- CodeProfile = "profile"
- CodeNoSeeders = "no_seeders"
- CodeDoubt = "doubt"
- // CodeNotBetter: the album, but no better than the copy the library
- // holds, for a want that is an upgrade.
- CodeNotBetter = "not_better"
-)
-
-// Reason is one thing a verdict rests on.
-type Reason struct {
- Kind string `json:"kind"`
- Code string `json:"code"`
- Text string `json:"text"`
-}
-
-// Want is the album searched for: its wants row and how the catalogue
-// describes it.
-type Want struct {
- ID pgtype.UUID
- release.Want
- // Floor is the tier of the copy the library holds, for an album wanted
- // for a better one: a release must rank above it. "" for an
- // album not held, and for a video rip, which any clean copy beats.
- Floor release.Tier
- // Held says the floor in words: "MP3, about 192 kbps".
- Held string
-}
-
-// Candidate is one result with everything decided about it.
-type Candidate struct {
- release.Release
- // Source is the indexer's own result. It never leaves the server: its
- // DownloadURL can carry the indexer's key.
- Source indexer.Result
- InfoHash string
- Indexer string
- SizeBytes int64
- Seeders int
- SwarmKnown bool
- Category int
- FilesCount int
- // Files is the torrent's file list once its .torrent has been read.
- Files []release.File
- FoldedFrom []string
-
- Tier release.Tier
- YearAgrees bool
- Verdict Verdict
- Reason string
- Reasons []Reason
-}
-
-// Key is how a candidate is named across a search and a grab: its infohash,
-// or its name when the source gave none.
-func Key(c Candidate) string {
- if c.InfoHash != "" {
- return c.InfoHash
- }
- return c.Name
-}
-
-// Result is one search.
-type Result struct {
- Query string
- // Unconfigured: no search source is set up.
- Unconfigured bool
- // Error is a search that failed outright.
- Error string
- // Narrowed names sources that did not answer on a search that otherwise
- // did.
- Narrowed []Narrowing
- Considered int
- Folded int
- // Looked is how many .torrent files were read for their file lists.
- Looked int
- Candidates []Candidate
- // KeepError: the candidates could not be stored.
- KeepError string
-}
-
-// Grabbable is what may be taken, best first.
-func (r Result) Grabbable() []Candidate {
- var out []Candidate
- for _, c := range r.Candidates {
- if c.Verdict == Grab {
- out = append(out, c)
- }
- }
- return out
-}
-
-// ForAPerson says whether anything found is worth a person's look: a
-// release that may be the album, whatever its quality.
-func (r Result) ForAPerson() bool {
- for _, c := range r.Candidates {
- if c.Verdict == Grab || c.Verdict == Review || c.Verdict == Refused {
- return true
- }
- }
- return false
-}
-
-// Service searches and grabs.
-type Service struct {
- // Search is nil when no source is configured.
- Search Searcher
- // Grabber is nil when no download client is configured.
- Grabber *Downloader
- Profile release.Profile
- // Memory stores candidates, grabs and blocks. Nil keeps nothing.
- Memory *Memory
-}
-
-// Query is the text sent to the indexers: the artist and the title, without
-// a trailing bracketed part the release names usually leave off.
-func Query(w release.Want) string {
- title := w.Title
- if i := strings.IndexAny(title, "(["); i > 0 {
- title = title[:i]
- }
- return strings.Join(strings.Fields(strings.TrimSpace(w.Artist)+" "+strings.TrimSpace(title)), " ")
-}
-
-// For searches for one want and judges everything that comes back. It never
-// returns an error: a search that could not be made is a Result saying so.
-func (s *Service) For(ctx context.Context, want Want) Result {
- r := Result{Query: Query(want.Want)}
- if s == nil || s.Search == nil {
- r.Unconfigured = true
- return r
- }
- found, err := s.Search.Search(ctx, indexer.Query{Text: r.Query, Categories: indexer.MusicCategories})
- // A partial search arrives as results and an error together.
- if narrowed, answered := narrowings(err); narrowed != nil && answered {
- r.Narrowed, err = narrowed, nil
- }
- switch {
- case errors.Is(err, indexer.ErrUnconfigured):
- r.Unconfigured = true
- return r
- case err != nil:
- r.Error = indexer.FirstLine(err.Error())
- return r
- }
- r.Considered = len(found)
- cands, folded := fold(found)
- r.Folded = folded
-
- blocked := s.Memory.blocked(ctx)
- for i := range cands {
- decide(&cands[i], want, s.Profile, blocked)
- }
- sortCandidates(cands)
- for _, i := range s.peek(ctx, cands) {
- decide(&cands[i], want, s.Profile, blocked)
- r.Looked++
- }
- sortCandidates(cands)
- r.Candidates = cands
- if s.Memory != nil && want.ID.Valid {
- if err := s.Memory.Keep(ctx, want.ID, cands); err != nil {
- r.KeepError = indexer.FirstLine(err.Error())
- }
- }
- return r
-}
-
-// decide works out one candidate's verdict, in order: given up on, then
-// whether it is the album, then whether the profile takes its quality, then
-// whether anything leaves it to a person. A quality refusal said about the
-// wrong album would be noise, so identity comes first.
-func decide(c *Candidate, want Want, profile release.Profile, blocked map[string]string) {
- w := want.Want
- c.Verdict, c.Reason, c.Reasons = "", "", nil
- if why, ok := blocked[c.InfoHash]; ok && c.InfoHash != "" {
- set(c, Blocked, Hard, CodeBlocked, why)
- return
- }
- m := release.Judge(w, c.Release)
- c.YearAgrees = m.YearAgrees
- if m.Veto != "" {
- set(c, Rejected, Hard, m.Veto, m.Reason)
- return
- }
- doubt := m.Doubt
- if len(c.Files) > 0 {
- if in := release.Inspect(c.Files); in.Format != "" {
- c.Format = in.Format
- }
- veto, reason, fileDoubt := release.JudgeFiles(w, c.Files)
- if veto != "" {
- set(c, Rejected, Hard, veto, reason)
- return
- }
- if doubt == "" {
- doubt = fileDoubt
- }
- }
- c.Tier = release.TierOf(c.Release, c.Category)
- if !profile.Allows(c.Tier) {
- set(c, Refused, Soft, CodeProfile, "Your quality settings don’t take "+strings.ToLower(release.Label(c.Tier))+".")
- return
- }
- if want.Floor != "" && release.Rank(c.Tier) <= release.Rank(want.Floor) {
- held := want.Held
- if held == "" {
- held = strings.ToLower(release.Label(want.Floor))
- }
- set(c, Refused, Soft, CodeNotBetter, "No better than the copy you have ("+held+").")
- return
- }
- if doubt != "" {
- set(c, Review, Soft, CodeDoubt, doubt)
- return
- }
- if c.Seeders == 0 {
- set(c, Review, Soft, CodeNoSeeders, "Nobody is sharing it right now.")
- return
- }
- c.Verdict = Grab
- c.Reason = "The album, " + strings.ToLower(release.Label(c.Tier)) + "."
- if d := release.Describe(c.Release); d != "" {
- c.Reason = "The album, " + d + "."
- }
-}
-
-func set(c *Candidate, v Verdict, kind, code, text string) {
- c.Verdict, c.Reason = v, text
- c.Reasons = append(c.Reasons, Reason{Kind: kind, Code: code, Text: text})
-}
-
-// sortCandidates puts the best first, stably so the same search proposes the
-// same grab twice.
-func sortCandidates(cands []Candidate) {
- sort.SliceStable(cands, func(a, b int) bool { return better(cands[a], cands[b]) })
-}
-
-// better says whether x ranks above y: verdict, then quality, then how many
-// share it, then whether its year agrees.
-func better(x, y Candidate) bool {
- if order[x.Verdict] != order[y.Verdict] {
- return order[x.Verdict] < order[y.Verdict]
- }
- if rx, ry := release.Rank(x.Tier), release.Rank(y.Tier); rx != ry {
- return rx > ry
- }
- if x.Seeders != y.Seeders {
- return x.Seeders > y.Seeders
- }
- return x.YearAgrees && !y.YearAgrees
-}
-
-// fold merges results that are the same torrent, by infohash. Prowlarr
-// aggregates, so one release arrives once per tracker that lists it. A
-// result without a hash is never folded: an absent hash is not a shared one.
-func fold(found []indexer.Result) ([]Candidate, int) {
- out := make([]Candidate, 0, len(found))
- at := make(map[string]int, len(found))
- folded := 0
- for _, f := range found {
- if strings.TrimSpace(f.Title) == "" {
- continue
- }
- hash := indexer.NormalizeHash(f.InfoHash)
- if hash == "" && f.Magnet != "" {
- hash = indexer.MagnetHash(f.Magnet)
- }
- if hash != "" {
- if i, seen := at[hash]; seen {
- folded++
- if f.Indexer != "" {
- out[i].FoldedFrom = append(out[i].FoldedFrom, f.Indexer)
- }
- // A measured swarm beats a reported one; among like, the
- // highest count.
- switch {
- case f.SwarmKnown && !out[i].SwarmKnown:
- out[i].Seeders, out[i].SwarmKnown = f.Seeders, true
- case f.SwarmKnown == out[i].SwarmKnown && f.Seeders > out[i].Seeders:
- out[i].Seeders = f.Seeders
- }
- if f.FilesCount > 0 && out[i].FilesCount == 0 {
- out[i].FilesCount = f.FilesCount
- }
- continue
- }
- at[hash] = len(out)
- }
- out = append(out, Candidate{
- Release: release.Parse(f.Title), Source: f, InfoHash: hash, Indexer: f.Indexer,
- SizeBytes: f.SizeBytes, Seeders: f.Seeders, SwarmKnown: f.SwarmKnown,
- Category: f.Category, FilesCount: f.FilesCount,
- })
- }
- return out, folded
-}
-
-// maxPeeks is how many .torrent files one search reads. Few: each goes
-// through Prowlarr to a tracker, and Prowlarr answers 429 to a burst.
-const maxPeeks = 3
-
-// peek reads the .torrent of candidates that are the album but whose name
-// states no format, so their files can say what they are; it returns which
-// now carry a file list. A failed fetch leaves the candidate as it was.
-func (s *Service) peek(ctx context.Context, cands []Candidate) []int {
- f, ok := s.Search.(Fetcher)
- if !ok {
- return nil
- }
- var looked []int
- for i := range cands {
- if len(looked) >= maxPeeks || ctx.Err() != nil {
- break
- }
- c := &cands[i]
- if c.Tier != release.TierUnknown || (c.Verdict != Grab && c.Verdict != Refused) ||
- c.Files != nil || c.Source.DownloadURL == "" || c.Source.Magnet != "" {
- continue
- }
- body, err := f.Download(ctx, c.Source)
- if err != nil {
- continue
- }
- meta, err := release.ParseTorrent(body)
- if err != nil {
- continue
- }
- c.Files = meta.Files
- if c.InfoHash == "" {
- c.InfoHash = meta.InfoHash
- }
- if c.FilesCount == 0 {
- c.FilesCount = len(meta.Files)
- }
- looked = append(looked, i)
- }
- return looked
-}
-
-// pick finds a candidate by key: the infohash first, the exact name for a
-// result that has none.
-func pick(cands []Candidate, key string) *Candidate {
- key = strings.TrimSpace(key)
- if key == "" {
- return nil
- }
- hash := indexer.NormalizeHash(key)
- for i := range cands {
- if cands[i].InfoHash != "" && cands[i].InfoHash == hash {
- return &cands[i]
- }
- }
- for i := range cands {
- if cands[i].InfoHash == "" && cands[i].Name == key {
- return &cands[i]
- }
- }
- return nil
-}
diff --git a/internal/acquire/search_test.go b/internal/acquire/search_test.go
deleted file mode 100644
index 63022d0e..00000000
--- a/internal/acquire/search_test.go
+++ /dev/null
@@ -1,291 +0,0 @@
-package acquire
-
-import (
- "context"
- "errors"
- "fmt"
- "strings"
- "testing"
- "time"
-
- "git.fabledsword.com/bvandeusen/minstrel/internal/downloader"
- "git.fabledsword.com/bvandeusen/minstrel/internal/indexer"
- "git.fabledsword.com/bvandeusen/minstrel/internal/release"
-)
-
-type fakeSearch struct {
- results []indexer.Result
- err error
- asked []indexer.Query
-}
-
-func (f *fakeSearch) Search(_ context.Context, q indexer.Query) ([]indexer.Result, error) {
- f.asked = append(f.asked, q)
- return f.results, f.err
-}
-
-var ram = Want{Want: release.Want{
- Artist: "Daft Punk", Title: "Random Access Memories", Year: 2013, PrimaryType: "Album",
- MinTracks: 13, MaxTracks: 15,
-}}
-
-var anyQuality = release.NewProfile([]string{
- "lossless_hires", "lossless", "lossy_high", "lossy_standard", "lossy_low", "lossy_poor", "lossy_trash",
-})
-
-func result(title, hash string, seeders int) indexer.Result {
- return indexer.Result{Title: title, InfoHash: hash, Seeders: seeders, Indexer: "Knaben", Source: "prowlarr",
- Magnet: "magnet:?xt=urn:btih:" + hash}
-}
-
-func TestFor_JudgesSortsAndFolds(t *testing.T) {
- search := &fakeSearch{results: []indexer.Result{
- result("Daft Punk - Random Access Memories (2013) [MP3 320]", "aaaa", 40),
- result("Daft Punk Random Access Memories (Drumless Edition 2023) [Flac 24-88]", "bbbb", 90),
- result("Daft Punk - Random Access Memories (2013) [FLAC]", "cccc", 12),
- result("Daft Punk - Random Access Memories (2013) [FLAC]", "CCCC", 30), // the same torrent from another tracker
- result("Daft Punk - Discovery (2001) [FLAC]", "dddd", 50),
- }}
- s := &Service{Search: search, Profile: anyQuality}
- r := s.For(context.Background(), ram)
-
- if len(search.asked) != 1 || search.asked[0].Text != "Daft Punk Random Access Memories" {
- t.Fatalf("asked %+v", search.asked)
- }
- if r.Considered != 5 || r.Folded != 1 || len(r.Candidates) != 4 {
- t.Fatalf("considered %d folded %d candidates %d", r.Considered, r.Folded, len(r.Candidates))
- }
- // Lossless before high-quality lossy among what can be taken; the
- // folded copy keeps the higher seeder count.
- first, second := r.Candidates[0], r.Candidates[1]
- if first.InfoHash != "cccc" || first.Verdict != Grab || first.Seeders != 30 || first.Tier != release.TierLossless {
- t.Errorf("first = %s %s %d %s", first.Name, first.Verdict, first.Seeders, first.Tier)
- }
- if second.InfoHash != "aaaa" || second.Verdict != Grab {
- t.Errorf("second = %s %s", second.Name, second.Verdict)
- }
- for _, c := range r.Candidates[2:] {
- if c.Verdict != Rejected || len(c.Reasons) != 1 || c.Reasons[0].Kind != Hard {
- t.Errorf("%s: %s %+v", c.Name, c.Verdict, c.Reasons)
- }
- }
- if len(r.Grabbable()) != 2 || !r.ForAPerson() {
- t.Errorf("grabbable %d", len(r.Grabbable()))
- }
-}
-
-func TestFor_ProfileSeedersAndDoubt(t *testing.T) {
- search := &fakeSearch{results: []indexer.Result{
- result("Daft Punk - Random Access Memories [MP3 128]", "aaaa", 5),
- result("Daft Punk - Random Access Memories [FLAC]", "bbbb", 0),
- result("Daft Punk - Random Access Memories", "cccc", 9),
- }}
- s := &Service{Search: search, Profile: release.NewProfile([]string{"lossless", "lossy_high"})}
- r := s.For(context.Background(), ram)
- verdicts := map[string]Verdict{}
- for _, c := range r.Candidates {
- verdicts[c.InfoHash] = c.Verdict
- }
- if verdicts["aaaa"] != Refused || verdicts["bbbb"] != Review || verdicts["cccc"] != Refused {
- t.Errorf("verdicts = %v", verdicts)
- }
- if len(r.Grabbable()) != 0 || !r.ForAPerson() {
- t.Error("nothing grabbable, but something for a person")
- }
-}
-
-func TestFor_BlockedReleasesAreSetAside(t *testing.T) {
- c := Candidate{Release: release.Parse("Daft Punk - Random Access Memories [FLAC]"), InfoHash: "aaaa", Seeders: 3}
- decide(&c, ram, anyQuality, map[string]string{"aaaa": "You set this release aside."})
- if c.Verdict != Blocked || c.Reason != "You set this release aside." {
- t.Errorf("verdict %s %q", c.Verdict, c.Reason)
- }
-}
-
-func TestFor_UnconfiguredAndFailedSearches(t *testing.T) {
- if r := (&Service{}).For(context.Background(), ram); !r.Unconfigured {
- t.Error("no source is unconfigured")
- }
- r := (&Service{Search: &fakeSearch{err: indexer.ErrUnconfigured}}).For(context.Background(), ram)
- if !r.Unconfigured {
- t.Error("ErrUnconfigured is unconfigured")
- }
- r = (&Service{Search: &fakeSearch{err: errors.New("dial tcp: refused")}}).For(context.Background(), ram)
- if r.Error == "" {
- t.Error("a failed search says so")
- }
-}
-
-func TestFanout_ANarrowedSearchKeepsWhatAnswered(t *testing.T) {
- up := &fakeSearch{results: []indexer.Result{result("Daft Punk - Random Access Memories [FLAC]", "aaaa", 3)}}
- down := &fakeSearch{err: indexer.ErrUnreachable}
- f := &Fanout{Sources: []Source{{Code: "prowlarr", Label: "Prowlarr", Search: up}, {Code: "torznab", Label: "Feed", Search: down}}}
- r := (&Service{Search: f, Profile: anyQuality}).For(context.Background(), ram)
- if r.Error != "" || len(r.Candidates) != 1 || len(r.Narrowed) != 1 || r.Narrowed[0].Source != "torznab" {
- t.Errorf("narrowed search: error %q, %d candidates, narrowed %+v", r.Error, len(r.Candidates), r.Narrowed)
- }
- f.Sources = f.Sources[1:]
- if r := (&Service{Search: f}).For(context.Background(), ram); r.Error == "" {
- t.Error("every source down is a failed search")
- }
-}
-
-func TestPacer_SpacesSearchesAndHoldsAfterA429(t *testing.T) {
- now := time.Unix(0, 0)
- var slept time.Duration
- p := &Pacer{Now: func() time.Time { return now }, Sleep: func(_ context.Context, d time.Duration) error { slept += d; return nil }}
- src := []Source{{Code: "prowlarr", PerHour: 120, Search: &fakeSearch{}}}
- p.spend(src[0])
- if !p.Wait(context.Background(), src, now.Add(time.Hour)) || slept != 30*time.Second {
- t.Errorf("slept %s, want 30s", slept)
- }
- if p.Wait(context.Background(), src, now.Add(time.Second)) {
- t.Error("waited past the deadline")
- }
- p.hold("prowlarr", &indexer.RateLimited{RetryAfter: 5 * time.Minute})
- if at := p.readyAt(src); at.Sub(now) != 5*time.Minute {
- t.Errorf("held until %s", at.Sub(now))
- }
-}
-
-type fakeClient struct {
- downloader.Driver
- added []downloader.Add
- addErr error
- deleted []string
- progress float64
- // elsewhere: the client holds it but does not list it for Minstrel.
- elsewhere bool
-}
-
-func (c *fakeClient) AddTorrent(_ context.Context, a downloader.Add) error {
- c.added = append(c.added, a)
- return c.addErr
-}
-
-func (c *fakeClient) Torrents(_ context.Context, hashes ...string) ([]downloader.Torrent, error) {
- if c.elsewhere {
- return nil, nil
- }
- return []downloader.Torrent{{Hash: hashes[0], Progress: c.progress}}, nil
-}
-
-func (c *fakeClient) Delete(_ context.Context, _ bool, hashes ...string) error {
- c.deleted = append(c.deleted, hashes...)
- return nil
-}
-
-func TestGrab_AddsAMagnetToItsOwnFolder(t *testing.T) {
- client := &fakeClient{}
- s := &Service{Search: &fakeSearch{results: []indexer.Result{
- result("Daft Punk - Random Access Memories (2013) [FLAC]", "ABCD", 8),
- }}, Profile: anyQuality, Grabber: &Downloader{Client: client, Code: "qbittorrent", Landing: "/downloads"}}
- r := s.For(context.Background(), ram)
- took := s.Grab(context.Background(), ram, &r.Candidates[0], r.Candidates, false, Origin{By: OriginSchedule})
- if !took.Added || took.SavePath != "/downloads/abcd" {
- t.Fatalf("took %+v", took)
- }
- if len(client.added) != 1 || !strings.Contains(strings.ToLower(client.added[0].Magnet), "abcd") || client.added[0].Tags[0] != Tag {
- t.Errorf("added %+v", client.added)
- }
-}
-
-func TestGrab_RefusesWhatTheVerdictWouldNotTake(t *testing.T) {
- client := &fakeClient{}
- s := &Service{Grabber: &Downloader{Client: client, Landing: "/downloads"}}
- review := Candidate{Verdict: Review, InfoHash: "aaaa", Source: indexer.Result{Magnet: "magnet:?xt=urn:btih:aaaa"}}
- if took := s.Grab(context.Background(), ram, &review, nil, false, Origin{By: OriginSchedule}); !took.Refused {
- t.Error("a review was taken unforced")
- }
- if took := s.Grab(context.Background(), ram, &review, nil, true, Origin{By: OriginOperator}); !took.Added {
- t.Errorf("a forced review was not taken: %+v", took)
- }
- blocked := Candidate{Verdict: Blocked, InfoHash: "bbbb"}
- if took := s.Grab(context.Background(), ram, &blocked, nil, true, Origin{By: OriginOperator}); !took.Refused {
- t.Error("forcing took a release set aside")
- }
- if took := (&Service{}).Grab(context.Background(), ram, &review, nil, true, Origin{}); !took.Unconfigured || took.Missing != MissingClient {
- t.Errorf("no client: %+v", took)
- }
-}
-
-func TestGrab_AClientThatAlreadyHasItCounts(t *testing.T) {
- client := &fakeClient{addErr: downloader.ErrDuplicate}
- s := &Service{Grabber: &Downloader{Client: client, Landing: "/downloads"}}
- c := Candidate{Verdict: Grab, InfoHash: "aaaa", Source: indexer.Result{Magnet: "magnet:?xt=urn:btih:aaaa"}}
- if took := s.Grab(context.Background(), ram, &c, nil, false, Origin{By: OriginSchedule}); !took.Added {
- t.Errorf("a duplicate: %+v", took)
- }
-}
-
-// A duplicate the client does not list for Minstrel sits under another
-// category, where it would read as gone half an hour later. It is set
-// aside rather than taken; forced, it is refused with the client's reason.
-func TestGrab_ADuplicateHeldOutsideMinstrelIsNotTaken(t *testing.T) {
- client := &fakeClient{addErr: fmt.Errorf("%w: already downloading Ohio Players (lidarr)", downloader.ErrDuplicate), elsewhere: true}
- s := &Service{Grabber: &Downloader{Client: client, Landing: "/downloads"}}
- c := Candidate{Verdict: Grab, InfoHash: "aaaa", Source: indexer.Result{Magnet: "magnet:?xt=urn:btih:aaaa"}}
- took := s.Grab(context.Background(), ram, &c, nil, false, Origin{By: OriginSchedule})
- if took.Added || !took.Refused {
- t.Fatalf("took %+v", took)
- }
- if c.Verdict != Blocked || !strings.Contains(c.Reason, "(already downloading Ohio Players (lidarr))") {
- t.Errorf("candidate %s: %q", c.Verdict, c.Reason)
- }
-
- forced := Candidate{Verdict: Grab, InfoHash: "aaaa", Source: indexer.Result{Magnet: "magnet:?xt=urn:btih:aaaa"}}
- took = s.Grab(context.Background(), ram, &forced, nil, true, Origin{By: OriginOperator})
- if took.Added || !strings.Contains(took.Error, "outside Minstrel's category") {
- t.Errorf("forced: %+v", took)
- }
-}
-
-func TestBetter_QualityThenSeedersThenYear(t *testing.T) {
- flac := Candidate{Verdict: Grab, Tier: release.TierLossless, Seeders: 2}
- mp3 := Candidate{Verdict: Grab, Tier: release.TierLossyHigh, Seeders: 90}
- if !better(flac, mp3) || better(mp3, flac) {
- t.Error("quality first among grabs")
- }
- busy := Candidate{Verdict: Grab, Tier: release.TierLossless, Seeders: 20}
- if !better(busy, flac) {
- t.Error("then seeders")
- }
- dated := Candidate{Verdict: Grab, Tier: release.TierLossless, Seeders: 2, YearAgrees: true}
- if !better(dated, flac) {
- t.Error("then the year")
- }
- if better(Candidate{Verdict: Review}, flac) || !better(flac, Candidate{Verdict: Review}) {
- t.Error("a grab ranks above a review")
- }
-}
-
-func TestLandingFor_OneFolderPerTorrent(t *testing.T) {
- if got := landingFor("/downloads", "abcd", "x"); got != "/downloads/abcd" {
- t.Error(got)
- }
- if got := landingFor("/downloads", "", "Daft Punk: R.A.M. [FLAC]"); got != "/downloads/daft-punk-r-a-m--flac" {
- t.Error(got)
- }
-}
-
-func TestDecide_AnUpgradeTakesOnlyABetterCopy(t *testing.T) {
- up := ram
- up.Floor, up.Held = release.TierLossyLow, "MP3, about 192 kbps"
- same := Candidate{Release: release.Parse("Daft Punk - Random Access Memories [MP3 192]"), InfoHash: "aaaa", Seeders: 3}
- decide(&same, up, anyQuality, nil)
- if same.Verdict != Refused || same.Reasons[0].Code != CodeNotBetter ||
- same.Reason != "No better than the copy you have (MP3, about 192 kbps)." {
- t.Errorf("same quality: %s %q", same.Verdict, same.Reason)
- }
- flac := Candidate{Release: release.Parse("Daft Punk - Random Access Memories [FLAC]"), InfoHash: "bbbb", Seeders: 3}
- decide(&flac, up, anyQuality, nil)
- if flac.Verdict != Grab {
- t.Errorf("a FLAC over a 192 MP3: %s %q", flac.Verdict, flac.Reason)
- }
- // A video rip has no floor: any clean copy is better.
- rip := ram
- decide(&same, rip, anyQuality, nil)
- if same.Verdict != Grab {
- t.Errorf("no floor: %s %q", same.Verdict, same.Reason)
- }
-}
diff --git a/internal/acquire/settings.go b/internal/acquire/settings.go
deleted file mode 100644
index 37726d61..00000000
--- a/internal/acquire/settings.go
+++ /dev/null
@@ -1,215 +0,0 @@
-// Package acquire turns the configured indexers and download clients into
-// working clients.
-//
-// ONE BUILDER PER KIND. SourceFor makes a search source from a row and
-// DownloadClient makes a download client, path mapping included. The grab,
-// the importer, the stall watchdog and the settings page's Test all build
-// through them, so a Test that passes vouches for the path real traffic takes.
-// Clients are rebuilt from the row on every use, so a changed address or a new
-// key takes effect on the next call.
-package acquire
-
-import (
- "context"
- "errors"
- "fmt"
- "time"
-
- "github.com/jackc/pgx/v5"
- "github.com/jackc/pgx/v5/pgxpool"
-
- "git.fabledsword.com/bvandeusen/minstrel/internal/db/dbq"
- "git.fabledsword.com/bvandeusen/minstrel/internal/downloader"
- "git.fabledsword.com/bvandeusen/minstrel/internal/exchange"
- "git.fabledsword.com/bvandeusen/minstrel/internal/indexer"
- "git.fabledsword.com/bvandeusen/minstrel/internal/prowlarr"
- "git.fabledsword.com/bvandeusen/minstrel/internal/secret"
-)
-
-// Kinds: which list a row appears in.
-const (
- KindSearch = "search"
- KindDownload = "download"
-)
-
-// Jobs a row can do. An exchange row does each its switch allows; every
-// other row does the one its kind names.
-const (
- JobSearch = "search"
- JobDownload = "download"
- JobEnrichment = "enrichment"
-)
-
-// Protocols: which client a row builds.
-const (
- ProtocolProwlarr = "prowlarr"
- ProtocolTorznab = "torznab"
- ProtocolQBittorrent = "qbittorrent"
- ProtocolExchange = "exchange"
-)
-
-// DefaultCategory files everything Minstrel adds to a download client.
-const DefaultCategory = "minstrel"
-
-// addable is what a new row of a protocol starts as.
-type addable struct {
- kind string
- label string
- category string
- timeoutMs int32
- perHour int32
-}
-
-// catalogue is every protocol an admin can add. Exchange starts unpaced
-// (it searches its own database); the others at 120 searches an hour, which
-// keeps a Prowlarr's indexers under the rate most trackers allow.
-var catalogue = map[string]addable{
- ProtocolProwlarr: {kind: KindSearch, label: "Prowlarr", timeoutMs: 60000, perHour: 120},
- ProtocolTorznab: {kind: KindSearch, label: "Torznab feed", timeoutMs: 30000, perHour: 120},
- ProtocolQBittorrent: {kind: KindDownload, label: "qBittorrent", category: DefaultCategory, timeoutMs: 15000, perHour: 0},
- ProtocolExchange: {kind: KindDownload, label: "FabledExchange", category: DefaultCategory, timeoutMs: 30000, perHour: 0},
-}
-
-// Addable says whether protocol is one an admin can add.
-func Addable(protocol string) bool {
- _, ok := catalogue[protocol]
- return ok
-}
-
-// Searches says whether a row is a search source.
-func Searches(row dbq.AcquisitionService) bool {
- if row.Protocol == ProtocolExchange {
- return row.ServesSearch
- }
- return row.Kind == KindSearch
-}
-
-// Fetches says whether a row is a download client.
-func Fetches(row dbq.AcquisitionService) bool {
- if row.Protocol == ProtocolExchange {
- return row.ServesDownloads
- }
- return row.Kind == KindDownload
-}
-
-// Searcher is a search source.
-type Searcher interface {
- Search(ctx context.Context, q indexer.Query) ([]indexer.Result, error)
-}
-
-// Fetcher fetches a result's .torrent through the source that offered it.
-type Fetcher interface {
- Download(ctx context.Context, r indexer.Result) ([]byte, error)
-}
-
-// Source is one configured search source.
-type Source struct {
- Code string
- Label string
- // PerHour is the most searches the schedule sends it an hour; 0 = unpaced.
- PerHour int
- Search Searcher
- // Fetch is nil for a source whose results are all magnets.
- Fetch Fetcher
-}
-
-// SourceFor builds the search source a row describes.
-func SourceFor(row dbq.AcquisitionService) Source {
- src := Source{Code: row.Code, Label: row.Label, PerHour: int(row.SearchesPerHour)}
- timeout := msDuration(row.TimeoutMs)
- switch row.Protocol {
- case ProtocolProwlarr:
- c := &prowlarr.Client{Config: prowlarr.Config{
- Code: row.Code, BaseURL: row.BaseUrl, APIKey: row.ApiKey, Timeout: timeout,
- }}
- src.Search, src.Fetch = c, c
- case ProtocolExchange:
- src.Search = ExchangeFor(row)
- default:
- c := &indexer.Torznab{Config: indexer.TorznabConfig{
- Code: row.Code, FeedURL: row.BaseUrl, APIKey: row.ApiKey, Timeout: timeout,
- }}
- src.Search, src.Fetch = c, c
- }
- return src
-}
-
-// ExchangeFor builds the search and enrichment client for an exchange row.
-func ExchangeFor(row dbq.AcquisitionService) *exchange.Client {
- return &exchange.Client{
- Code: row.Code,
- Config: exchange.Config{
- BaseURL: row.BaseUrl, APIKey: row.ApiKey, Timeout: msDuration(row.TimeoutMs),
- },
- }
-}
-
-// DownloadClient builds the download client a row describes, wrapped in its
-// path mapping.
-func DownloadClient(row dbq.AcquisitionService) (downloader.Driver, error) {
- d, err := downloader.For(row.Protocol, downloader.Config{
- BaseURL: row.BaseUrl, Username: row.Username, Password: row.Password,
- APIKey: row.ApiKey, Category: row.Category, Timeout: msDuration(row.TimeoutMs),
- })
- if err != nil {
- return nil, err
- }
- return downloader.Mapped(d, downloader.PathMap{Local: row.LocalPath, Remote: row.RemotePath}), nil
-}
-
-// DownloadClientFor builds the download client a grab went to, by the code
-// of its acquisition_services row.
-func DownloadClientFor(ctx context.Context, pool *pgxpool.Pool, code string) (downloader.Driver, error) {
- row, err := getService(ctx, dbq.New(pool), code)
- if err != nil {
- if errors.Is(err, pgx.ErrNoRows) {
- return nil, fmt.Errorf("the download client %q is no longer set up", code)
- }
- return nil, err
- }
- return DownloadClient(row)
-}
-
-// Sources builds every enabled search source.
-func Sources(ctx context.Context, pool *pgxpool.Pool) ([]Source, error) {
- rows, err := usableServices(ctx, dbq.New(pool), JobSearch)
- if err != nil {
- return nil, err
- }
- out := make([]Source, len(rows))
- for i, r := range rows {
- out[i] = SourceFor(r)
- }
- return out, nil
-}
-
-// getService and usableServices are the only reads of a full
-// acquisition_services row. They hand back the credentials opened, so the
-// clients built from a row are given the key and password themselves.
-func getService(ctx context.Context, q *dbq.Queries, code string) (dbq.AcquisitionService, error) {
- row, err := q.GetAcquisitionService(ctx, code)
- if err != nil {
- return row, err
- }
- return revealed(row), nil
-}
-
-func usableServices(ctx context.Context, q *dbq.Queries, job string) ([]dbq.AcquisitionService, error) {
- rows, err := q.ListUsableAcquisitionServices(ctx, job)
- if err != nil {
- return nil, err
- }
- for i := range rows {
- rows[i] = revealed(rows[i])
- }
- return rows, nil
-}
-
-// revealed is row with its sealed credentials opened.
-func revealed(row dbq.AcquisitionService) dbq.AcquisitionService {
- row.ApiKey = secret.Reveal(secret.AcquisitionAPIKey.Purpose(), row.Code, row.ApiKey)
- row.Password = secret.Reveal(secret.AcquisitionPassword.Purpose(), row.Code, row.Password)
- return row
-}
-
-func msDuration(ms int32) time.Duration { return time.Duration(ms) * time.Millisecond }
diff --git a/internal/acquire/store.go b/internal/acquire/store.go
deleted file mode 100644
index b7141fda..00000000
--- a/internal/acquire/store.go
+++ /dev/null
@@ -1,200 +0,0 @@
-package acquire
-
-import (
- "context"
- "errors"
- "fmt"
- "net/url"
- "strings"
-
- "github.com/jackc/pgx/v5"
- "github.com/jackc/pgx/v5/pgxpool"
-
- "git.fabledsword.com/bvandeusen/minstrel/internal/db/dbq"
- "git.fabledsword.com/bvandeusen/minstrel/internal/secret"
-)
-
-// ErrNotFound is a code with no row.
-var ErrNotFound = errors.New("no such service")
-
-// ErrTooMany is a twenty-first row of one protocol.
-var ErrTooMany = errors.New("there are already twenty of these")
-
-// InvalidError is a setting an admin has to change, in words for the
-// settings card.
-type InvalidError struct {
- Field string
- Message string
-}
-
-func (e *InvalidError) Error() string { return e.Message }
-
-func invalid(field, msg string) error { return &InvalidError{Field: field, Message: msg} }
-
-// maxPerProtocol bounds the codes tried for a new row. Nobody runs twenty
-// Prowlarrs; the bound keeps a loop over a full table finite.
-const maxPerProtocol = 20
-
-// Store reads and writes acquisition_services.
-type Store struct {
- pool *pgxpool.Pool
-}
-
-// NewStore returns a Store over pool.
-func NewStore(pool *pgxpool.Pool) *Store { return &Store{pool: pool} }
-
-// List returns every row without its secrets.
-func (s *Store) List(ctx context.Context) ([]dbq.ListAcquisitionServicesRow, error) {
- return dbq.New(s.pool).ListAcquisitionServices(ctx)
-}
-
-// Get returns one row, secrets included: for building a client only.
-func (s *Store) Get(ctx context.Context, code string) (dbq.AcquisitionService, error) {
- row, err := getService(ctx, dbq.New(s.pool), code)
- if errors.Is(err, pgx.ErrNoRows) {
- return row, ErrNotFound
- }
- return row, err
-}
-
-// Create adds a row of protocol, switched off and with no address. Its code
-// is the protocol for the first, then protocol-2, protocol-3.
-func (s *Store) Create(ctx context.Context, protocol string) (dbq.AcquisitionService, error) {
- a, ok := catalogue[protocol]
- if !ok {
- return dbq.AcquisitionService{}, invalid("protocol", "One of prowlarr, torznab, qbittorrent, exchange.")
- }
- q := dbq.New(s.pool)
- for n := 1; n <= maxPerProtocol; n++ {
- code, label := protocol, a.label
- if n > 1 {
- code, label = fmt.Sprintf("%s-%d", protocol, n), fmt.Sprintf("%s %d", a.label, n)
- }
- row, err := q.CreateAcquisitionService(ctx, dbq.CreateAcquisitionServiceParams{
- Code: code, Kind: a.kind, Protocol: protocol, Label: label,
- Category: a.category, TimeoutMs: a.timeoutMs, SearchesPerHour: a.perHour,
- })
- if errors.Is(err, pgx.ErrNoRows) {
- continue
- }
- return row, err
- }
- return dbq.AcquisitionService{}, ErrTooMany
-}
-
-// Update is everything on the settings form except the credentials.
-type Update struct {
- Label string
- Enabled bool
- BaseURL string
- Username string
- Category string
- TimeoutMs int32
- ServesSearch bool
- ServesDownloads bool
- ServesEnrichment bool
- LocalPath string
- RemotePath string
- SearchesPerHour int32
-}
-
-// Save validates u and writes it to code's row.
-func (s *Store) Save(ctx context.Context, code string, u Update) (dbq.AcquisitionService, error) {
- u.Label = strings.TrimSpace(u.Label)
- u.BaseURL = strings.TrimSpace(u.BaseURL)
- u.Username = strings.TrimSpace(u.Username)
- u.Category = strings.TrimSpace(u.Category)
- u.LocalPath = strings.TrimSpace(u.LocalPath)
- u.RemotePath = strings.TrimSpace(u.RemotePath)
- if err := u.validate(); err != nil {
- return dbq.AcquisitionService{}, err
- }
- row, err := dbq.New(s.pool).UpdateAcquisitionService(ctx, dbq.UpdateAcquisitionServiceParams{
- Code: code, Label: u.Label, Enabled: u.Enabled, BaseUrl: u.BaseURL,
- Username: u.Username, Category: u.Category, TimeoutMs: u.TimeoutMs,
- ServesSearch: u.ServesSearch, ServesDownloads: u.ServesDownloads,
- ServesEnrichment: u.ServesEnrichment, LocalPath: u.LocalPath,
- RemotePath: u.RemotePath, SearchesPerHour: u.SearchesPerHour,
- })
- if errors.Is(err, pgx.ErrNoRows) {
- return row, ErrNotFound
- }
- return row, err
-}
-
-// validate mirrors the table's CHECKs, so a bad value is named in the card's
-// own terms before Postgres refuses it.
-func (u Update) validate() error {
- if u.Label == "" {
- return invalid("label", "Name: give it a name.")
- }
- if u.Enabled && u.BaseURL == "" {
- return invalid("base_url", "Address: give the address before switching this on.")
- }
- if u.BaseURL != "" {
- parsed, err := url.Parse(u.BaseURL)
- if err != nil || (parsed.Scheme != "http" && parsed.Scheme != "https") || parsed.Host == "" {
- return invalid("base_url", "Address: start with http:// or https://.")
- }
- }
- if u.TimeoutMs < 1000 || u.TimeoutMs > 300000 {
- return invalid("timeout_ms", "Give up after: between 1000 and 300000 milliseconds.")
- }
- if !strings.HasPrefix(u.LocalPath, "/") {
- return invalid("local_path", "Downloads folder as Minstrel sees it: a full path, starting with /.")
- }
- if u.RemotePath != "" && !strings.HasPrefix(u.RemotePath, "/") {
- return invalid("remote_path", "Downloads folder as the client sees it: a full path starting with /, or empty when it is the same path.")
- }
- if u.SearchesPerHour < 0 || u.SearchesPerHour > 3600 {
- return invalid("searches_per_hour", "Searches per hour: between 0 and 3600. 0 means no limit.")
- }
- return nil
-}
-
-// Delete removes code's row.
-func (s *Store) Delete(ctx context.Context, code string) error {
- n, err := dbq.New(s.pool).DeleteAcquisitionService(ctx, code)
- if err != nil {
- return err
- }
- if n == 0 {
- return ErrNotFound
- }
- return nil
-}
-
-// SetAPIKey stores a key, trimmed and sealed; empty clears it.
-func (s *Store) SetAPIKey(ctx context.Context, code, key string) error {
- sealed, err := secret.Seal(secret.AcquisitionAPIKey.Purpose(), code, strings.TrimSpace(key))
- if err != nil {
- return err
- }
- n, err := dbq.New(s.pool).SetAcquisitionServiceAPIKey(ctx, dbq.SetAcquisitionServiceAPIKeyParams{
- Code: code, ApiKey: sealed,
- })
- return affected(n, err)
-}
-
-// SetPassword stores a password as typed (spaces are part of a password),
-// sealed; empty clears it.
-func (s *Store) SetPassword(ctx context.Context, code, password string) error {
- sealed, err := secret.Seal(secret.AcquisitionPassword.Purpose(), code, password)
- if err != nil {
- return err
- }
- n, err := dbq.New(s.pool).SetAcquisitionServicePassword(ctx, dbq.SetAcquisitionServicePasswordParams{
- Code: code, Password: sealed,
- })
- return affected(n, err)
-}
-
-func affected(n int64, err error) error {
- if err != nil {
- return err
- }
- if n == 0 {
- return ErrNotFound
- }
- return nil
-}
diff --git a/internal/acquire/store_test.go b/internal/acquire/store_test.go
deleted file mode 100644
index 420a38b0..00000000
--- a/internal/acquire/store_test.go
+++ /dev/null
@@ -1,118 +0,0 @@
-package acquire
-
-import (
- "context"
- "errors"
- "io"
- "log/slog"
- "os"
- "testing"
-
- "github.com/jackc/pgx/v5/pgxpool"
-
- "git.fabledsword.com/bvandeusen/minstrel/internal/db"
- "git.fabledsword.com/bvandeusen/minstrel/internal/dbtest"
- "git.fabledsword.com/bvandeusen/minstrel/internal/secret"
-)
-
-func newPool(t *testing.T) *pgxpool.Pool {
- t.Helper()
- if testing.Short() {
- t.Skip("skipping integration test in -short mode")
- }
- dsn := os.Getenv("MINSTREL_TEST_DATABASE_URL")
- if dsn == "" {
- t.Skip("MINSTREL_TEST_DATABASE_URL not set")
- }
- if err := db.Migrate(dsn, slog.New(slog.NewTextHandler(io.Discard, nil))); err != nil {
- t.Fatalf("migrate: %v", err)
- }
- pool, err := pgxpool.New(context.Background(), dsn)
- if err != nil {
- t.Fatalf("pool: %v", err)
- }
- t.Cleanup(pool.Close)
- dbtest.ResetDB(t, pool)
- return pool
-}
-
-func TestStore_AddsConfiguresAndRemoves_Integration(t *testing.T) {
- ctx := context.Background()
- s := NewStore(newPool(t))
-
- first, err := s.Create(ctx, ProtocolExchange)
- if err != nil {
- t.Fatalf("create: %v", err)
- }
- second, err := s.Create(ctx, ProtocolExchange)
- if err != nil {
- t.Fatalf("create a second: %v", err)
- }
- if first.Code != "exchange" || second.Code != "exchange-2" || second.Label != "FabledExchange 2" {
- t.Errorf("codes = %q, %q (%q); want exchange, exchange-2", first.Code, second.Code, second.Label)
- }
- // A new exchange is off, does every job, files under minstrel and is
- // unpaced.
- if first.Enabled || first.Kind != KindDownload || !first.ServesSearch || !first.ServesDownloads ||
- !first.ServesEnrichment || first.Category != DefaultCategory || first.SearchesPerHour != 0 || first.LocalPath != "/downloads" {
- t.Errorf("a new exchange = %+v", first)
- }
- if _, err := s.Create(ctx, "transmission"); !errors.As(err, new(*InvalidError)) {
- t.Errorf("an unknown protocol: err = %v", err)
- }
-
- u := Update{
- Label: "Exchange", Enabled: true, BaseURL: "http://exchange:8080", Category: "minstrel",
- TimeoutMs: 20000, ServesSearch: true, LocalPath: "/downloads", RemotePath: "/data/torrents",
- }
- saved, err := s.Save(ctx, first.Code, u)
- if err != nil {
- t.Fatalf("save: %v", err)
- }
- if !saved.Enabled || saved.ServesDownloads || saved.RemotePath != "/data/torrents" {
- t.Errorf("saved = %+v", saved)
- }
- if _, err := s.Save(ctx, "nope", u); !errors.Is(err, ErrNotFound) {
- t.Errorf("saving a missing row: err = %v", err)
- }
-
- // Secrets are write-only: the list says whether one is set, never what.
- if err := s.SetAPIKey(ctx, first.Code, " key "); err != nil {
- t.Fatalf("set key: %v", err)
- }
- full, err := s.Get(ctx, first.Code)
- if err != nil || full.ApiKey != "key" {
- t.Errorf("stored key = %q (err %v), want it trimmed", full.ApiKey, err)
- }
- // The column holds the key sealed, never the key.
- var raw string
- if err := s.pool.QueryRow(ctx,
- "SELECT api_key FROM acquisition_services WHERE code = $1", first.Code).Scan(&raw); err != nil {
- t.Fatalf("read api_key: %v", err)
- }
- if raw == "key" || !secret.IsSealed(raw) {
- t.Errorf("api_key column = %q, want it sealed", raw)
- }
- rows, err := s.List(ctx)
- if err != nil || len(rows) != 2 {
- t.Fatalf("list: %d rows, err %v", len(rows), err)
- }
- for _, r := range rows {
- if r.Code == first.Code && !r.HasApiKey {
- t.Error("the list should say a key is set")
- }
- }
-
- // Only the exchange with search switched on is a search source.
- srcs, err := Sources(ctx, s.pool)
- if err != nil || len(srcs) != 1 || srcs[0].Code != first.Code {
- t.Errorf("sources = %+v, err %v; want only the enabled exchange", srcs, err)
- }
-
- if err := s.Delete(ctx, second.Code); err != nil {
- t.Fatalf("delete: %v", err)
- }
- if err := s.Delete(ctx, second.Code); !errors.Is(err, ErrNotFound) {
- t.Errorf("deleting twice: err = %v", err)
- }
-}
diff --git a/internal/acquire/test.go b/internal/acquire/test.go
deleted file mode 100644
index 9ec6d07b..00000000
--- a/internal/acquire/test.go
+++ /dev/null
@@ -1,134 +0,0 @@
-package acquire
-
-import (
- "context"
- "errors"
- "fmt"
- "os"
- "strings"
-
- "git.fabledsword.com/bvandeusen/minstrel/internal/db/dbq"
- "git.fabledsword.com/bvandeusen/minstrel/internal/exchange"
- "git.fabledsword.com/bvandeusen/minstrel/internal/indexer"
-)
-
-// TestResult is what the settings card says after Test.
-type TestResult struct {
- OK bool `json:"ok"`
- Detail string `json:"detail"`
-}
-
-// testQuery is a search every music indexer should answer.
-const testQuery = "Radiohead"
-
-// Test reaches the service a row describes, through the same builders real
-// traffic uses. It runs whether or not the row is switched on: paste the
-// address, test it, then switch it on.
-func Test(ctx context.Context, row dbq.AcquisitionService) TestResult {
- if strings.TrimSpace(row.BaseUrl) == "" {
- return TestResult{Detail: "No address is set, so there is nothing to reach yet."}
- }
- if row.Protocol == ProtocolExchange {
- return testExchange(ctx, row)
- }
- if row.Kind == KindSearch {
- return testSearch(ctx, row)
- }
- return testDownload(ctx, row)
-}
-
-// testExchange runs each job the row has switched on and reports them all.
-func testExchange(ctx context.Context, row dbq.AcquisitionService) TestResult {
- var parts []string
- ok := true
- run := func(name string, on bool, test func() TestResult) {
- if !on {
- return
- }
- r := test()
- ok = ok && r.OK
- parts = append(parts, name+": "+r.Detail)
- }
- run("Downloads", row.ServesDownloads, func() TestResult { return testDownload(ctx, row) })
- run("Search", row.ServesSearch, func() TestResult { return testSearch(ctx, row) })
- run("Release details", row.ServesEnrichment, func() TestResult { return testEnrichment(ctx, row) })
- if len(parts) == 0 {
- return TestResult{Detail: "Every job of this FabledExchange is switched off, so there is nothing to test."}
- }
- return TestResult{OK: ok, Detail: strings.Join(parts, " ")}
-}
-
-// testSearch runs a real search.
-func testSearch(ctx context.Context, row dbq.AcquisitionService) TestResult {
- src := SourceFor(row)
- results, err := src.Search.Search(ctx, indexer.Query{Text: testQuery, Categories: indexer.MusicCategories})
- switch {
- case errors.Is(err, indexer.ErrUnconfigured):
- return TestResult{Detail: "Add this service's address, and the API key it shows on its own settings page."}
- case errors.Is(err, indexer.ErrAuth):
- return TestResult{Detail: indexer.FirstLine(err.Error()) + ". Check it on the service's own settings page."}
- case err != nil:
- return TestResult{Detail: indexer.FirstLine(err.Error())}
- case len(results) == 0:
- return TestResult{OK: true, Detail: "Connected, but a test search for music found nothing. Check that it has an indexer with music categories."}
- }
- return TestResult{OK: true, Detail: fmt.Sprintf("Connected. %d results for a test search.", len(results))}
-}
-
-// testDownload lists the client's torrents, which proves the session and not
-// only the login, then checks the downloads folder from Minstrel's side.
-func testDownload(ctx context.Context, row dbq.AcquisitionService) TestResult {
- c, err := DownloadClient(row)
- if err != nil {
- return TestResult{Detail: err.Error()}
- }
- ts, err := c.Torrents(ctx)
- if err != nil {
- return TestResult{Detail: indexer.FirstLine(err.Error())}
- }
- detail := fmt.Sprintf("Connected. %d torrents.", len(ts))
- if row.Category != "" {
- detail = fmt.Sprintf("Connected. %d torrents in the %q category.", len(ts), row.Category)
- }
- if up, err := c.Connected(ctx); err == nil && !up {
- detail += " The client reports it is not connected to any peers, so downloads will stall."
- }
- if problem := landingProblem(row.LocalPath); problem != "" {
- return TestResult{Detail: detail + " " + problem}
- }
- return TestResult{OK: true, Detail: detail}
-}
-
-// landingProblem checks Minstrel can see and write the downloads folder: it
-// moves finished downloads out of it. Only the local path can be checked;
-// the client's own name for it cannot be seen from here.
-func landingProblem(dir string) string {
- info, err := os.Stat(dir)
- switch {
- case err != nil:
- return fmt.Sprintf("But Minstrel cannot see its downloads folder at %s. Mount the download client's downloads folder there, or change where Minstrel looks for it.", dir)
- case !info.IsDir():
- return fmt.Sprintf("But %s is a file, not the downloads folder.", dir)
- }
- probe, err := os.CreateTemp(dir, ".minstrel-probe-*")
- if err != nil {
- return fmt.Sprintf("But Minstrel cannot write to %s, so it could not move a finished download out of it. Check the mount is not read-only and Minstrel's user can write there.", dir)
- }
- name := probe.Name()
- _ = probe.Close()
- _ = os.Remove(name)
- return ""
-}
-
-// testEnrichment asks about a hash no torrent has. A JSON "not crawled" is a
-// healthy answer; a 404 from something that is not Exchange is not.
-func testEnrichment(ctx context.Context, row dbq.AcquisitionService) TestResult {
- _, err := ExchangeFor(row).Torrent(ctx, strings.Repeat("0", 40))
- switch {
- case err == nil, errors.Is(err, exchange.ErrUnknown):
- return TestResult{OK: true, Detail: "Connected. The index answered."}
- case errors.Is(err, exchange.ErrUnconfigured):
- return TestResult{Detail: "No address is set, so there is nothing to reach yet."}
- }
- return TestResult{Detail: indexer.FirstLine(err.Error())}
-}
diff --git a/internal/api/admin_acquisition.go b/internal/api/admin_acquisition.go
deleted file mode 100644
index f6144e61..00000000
--- a/internal/api/admin_acquisition.go
+++ /dev/null
@@ -1,210 +0,0 @@
-package api
-
-import (
- "encoding/json"
- "errors"
- "io"
- "net/http"
-
- "github.com/go-chi/chi/v5"
-
- "git.fabledsword.com/bvandeusen/minstrel/internal/acquire"
- "git.fabledsword.com/bvandeusen/minstrel/internal/apierror"
- "git.fabledsword.com/bvandeusen/minstrel/internal/db/dbq"
-)
-
-// Indexers and download clients, admin side. Rows an admin
-// adds and configures; nothing is seeded. Secrets are write-only: no response
-// carries one, only whether it is set.
-
-// acquisitionBodyLimit caps a settings body. The largest is a full form of
-// short strings.
-const acquisitionBodyLimit = 64 << 10
-
-type acquisitionServiceResp struct {
- Code string `json:"code"`
- Kind string `json:"kind"`
- Protocol string `json:"protocol"`
- Label string `json:"label"`
- Enabled bool `json:"enabled"`
- BaseURL string `json:"base_url"`
- HasAPIKey bool `json:"has_api_key"`
- HasPassword bool `json:"has_password"`
- Username string `json:"username"`
- Category string `json:"category"`
- TimeoutMs int32 `json:"timeout_ms"`
- ServesSearch bool `json:"serves_search"`
- ServesDownloads bool `json:"serves_downloads"`
- ServesEnrichment bool `json:"serves_enrichment"`
- LocalPath string `json:"local_path"`
- RemotePath string `json:"remote_path"`
- SearchesPerHour int32 `json:"searches_per_hour"`
- // Usable is switched on and addressed: the state the jobs read.
- Usable bool `json:"usable"`
-}
-
-func acquisitionRespOf(r dbq.AcquisitionService) acquisitionServiceResp {
- return acquisitionServiceResp{
- Code: r.Code, Kind: r.Kind, Protocol: r.Protocol, Label: r.Label,
- Enabled: r.Enabled, BaseURL: r.BaseUrl,
- HasAPIKey: r.ApiKey != "", HasPassword: r.Password != "",
- Username: r.Username, Category: r.Category, TimeoutMs: r.TimeoutMs,
- ServesSearch: r.ServesSearch, ServesDownloads: r.ServesDownloads,
- ServesEnrichment: r.ServesEnrichment,
- LocalPath: r.LocalPath, RemotePath: r.RemotePath,
- SearchesPerHour: r.SearchesPerHour,
- Usable: r.Enabled && r.BaseUrl != "",
- }
-}
-
-// handleListAcquisitionServices implements GET /api/admin/acquisition/services.
-func (h *handlers) handleListAcquisitionServices(w http.ResponseWriter, r *http.Request) {
- rows, err := h.acquisition.List(r.Context())
- if err != nil {
- writeErrWithLog(w, h.logger, "admin: list acquisition services", apierror.Internal(err))
- return
- }
- out := make([]acquisitionServiceResp, len(rows))
- for i, row := range rows {
- out[i] = acquisitionRespOf(dbq.AcquisitionService{
- Code: row.Code, Kind: row.Kind, Protocol: row.Protocol, Label: row.Label,
- Enabled: row.Enabled, BaseUrl: row.BaseUrl, Username: row.Username,
- Category: row.Category, TimeoutMs: row.TimeoutMs,
- ServesSearch: row.ServesSearch, ServesDownloads: row.ServesDownloads,
- ServesEnrichment: row.ServesEnrichment, LocalPath: row.LocalPath,
- RemotePath: row.RemotePath, SearchesPerHour: row.SearchesPerHour,
- })
- out[i].HasAPIKey, out[i].HasPassword = row.HasApiKey, row.HasPassword
- }
- writeJSON(w, http.StatusOK, map[string]any{"services": out})
-}
-
-// handleCreateAcquisitionService implements POST
-// /api/admin/acquisition/services with {"protocol":
-// "prowlarr"|"torznab"|"qbittorrent"|"exchange"}. The row starts switched off,
-// with no address.
-func (h *handlers) handleCreateAcquisitionService(w http.ResponseWriter, r *http.Request) {
- var body struct {
- Protocol string `json:"protocol"`
- }
- if !decodeAcquisitionBody(w, r, &body) {
- return
- }
- row, err := h.acquisition.Create(r.Context(), body.Protocol)
- if err != nil {
- h.writeAcquisitionErr(w, "admin: create acquisition service", err)
- return
- }
- writeJSON(w, http.StatusCreated, acquisitionRespOf(row))
-}
-
-type updateAcquisitionBody struct {
- Label string `json:"label"`
- Enabled bool `json:"enabled"`
- BaseURL string `json:"base_url"`
- Username string `json:"username"`
- Category string `json:"category"`
- TimeoutMs int32 `json:"timeout_ms"`
- ServesSearch bool `json:"serves_search"`
- ServesDownloads bool `json:"serves_downloads"`
- ServesEnrichment bool `json:"serves_enrichment"`
- LocalPath string `json:"local_path"`
- RemotePath string `json:"remote_path"`
- SearchesPerHour int32 `json:"searches_per_hour"`
-}
-
-// handleUpdateAcquisitionService implements PUT
-// /api/admin/acquisition/services/{code}: the whole form except credentials.
-func (h *handlers) handleUpdateAcquisitionService(w http.ResponseWriter, r *http.Request) {
- var body updateAcquisitionBody
- if !decodeAcquisitionBody(w, r, &body) {
- return
- }
- row, err := h.acquisition.Save(r.Context(), chi.URLParam(r, "code"), acquire.Update(body))
- if err != nil {
- h.writeAcquisitionErr(w, "admin: update acquisition service", err)
- return
- }
- writeJSON(w, http.StatusOK, acquisitionRespOf(row))
-}
-
-// handleDeleteAcquisitionService implements DELETE
-// /api/admin/acquisition/services/{code}.
-func (h *handlers) handleDeleteAcquisitionService(w http.ResponseWriter, r *http.Request) {
- if err := h.acquisition.Delete(r.Context(), chi.URLParam(r, "code")); err != nil {
- h.writeAcquisitionErr(w, "admin: delete acquisition service", err)
- return
- }
- w.WriteHeader(http.StatusNoContent)
-}
-
-// handleTestAcquisitionService implements POST
-// /api/admin/acquisition/services/{code}/test. POST because it reaches
-// another machine, which a prefetch must not trigger. Always 200 with
-// {ok, detail}: a failed connection is the answer, not an error.
-func (h *handlers) handleTestAcquisitionService(w http.ResponseWriter, r *http.Request) {
- row, err := h.acquisition.Get(r.Context(), chi.URLParam(r, "code"))
- if err != nil {
- h.writeAcquisitionErr(w, "admin: test acquisition service", err)
- return
- }
- writeJSON(w, http.StatusOK, acquire.Test(r.Context(), row))
-}
-
-// handleSetAcquisitionSecret implements PUT and DELETE on
-// /api/admin/acquisition/services/{code}/api-key and .../password. PUT takes
-// {"value": "..."}; DELETE clears it. 204 with no body either way: echoing
-// anything here is how a secret ends up in a response.
-func (h *handlers) handleSetAcquisitionSecret(field string) http.HandlerFunc {
- return func(w http.ResponseWriter, r *http.Request) {
- value := ""
- if r.Method == http.MethodPut {
- var body struct {
- Value string `json:"value"`
- }
- if !decodeAcquisitionBody(w, r, &body) {
- return
- }
- if body.Value == "" {
- writeErr(w, apierror.BadRequest("invalid_setting", "Paste a value, or use Remove to clear the one that is set."))
- return
- }
- value = body.Value
- }
- code := chi.URLParam(r, "code")
- var err error
- if field == "password" {
- err = h.acquisition.SetPassword(r.Context(), code, value)
- } else {
- err = h.acquisition.SetAPIKey(r.Context(), code, value)
- }
- if err != nil {
- h.writeAcquisitionErr(w, "admin: set acquisition "+field, err)
- return
- }
- w.WriteHeader(http.StatusNoContent)
- }
-}
-
-func decodeAcquisitionBody(w http.ResponseWriter, r *http.Request, into any) bool {
- err := json.NewDecoder(http.MaxBytesReader(w, r.Body, acquisitionBodyLimit)).Decode(into)
- if err != nil && !errors.Is(err, io.EOF) {
- writeErr(w, apierror.BadRequest("invalid_body", "malformed JSON"))
- return false
- }
- return true
-}
-
-func (h *handlers) writeAcquisitionErr(w http.ResponseWriter, msg string, err error) {
- var invalid *acquire.InvalidError
- switch {
- case errors.As(err, &invalid):
- writeErr(w, apierror.BadRequest("invalid_setting", invalid.Message))
- case errors.Is(err, acquire.ErrNotFound):
- writeErr(w, apierror.NotFound("service"))
- case errors.Is(err, acquire.ErrTooMany):
- writeErr(w, apierror.Conflict("too_many_services", "There are already twenty of these. Remove one you no longer use first."))
- default:
- writeErrWithLog(w, h.logger, msg, apierror.Internal(err))
- }
-}
diff --git a/internal/api/admin_duplicates.go b/internal/api/admin_duplicates.go
index 970478f6..b43cea18 100644
--- a/internal/api/admin_duplicates.go
+++ b/internal/api/admin_duplicates.go
@@ -459,21 +459,19 @@ type refetchDuplicateRequest struct {
KeepTrackID string `json:"keep_track_id"`
}
-// refetchDuplicateResponse reports what went. Blocked names the releases set
-// aside; empty when the album came from before Minstrel downloaded anything.
+// refetchDuplicateResponse reports what went.
type refetchDuplicateResponse struct {
KeptTrackID *string `json:"kept_track_id"`
RemovedPaths []string `json:"removed_paths"`
- Blocked []string `json:"blocked"`
}
// handleRefetchDuplicateGroup implements POST
// /api/admin/library/duplicates/{id}/refetch: settle a group of one
// recording in several slots of an album the way the resolver would, with an
// admin's answer. The copies in slots that are not the recording's are
-// removed and marked missing, the release they came from is set aside, and the
-// album is wanted again. Merge is the wrong action there: it keeps one title
-// whether or not it is the audio's, and fetches nothing.
+// removed and marked missing, so the slot shows as missing until a good copy
+// arrives in the library. Merge is the wrong action there: it keeps one title
+// whether or not it is the audio's.
//
// Errors:
// - 409 library_not_writable / 500 file_delete_failed when a file could not be
@@ -525,10 +523,7 @@ func (h *handlers) handleRefetchDuplicateGroup(w http.ResponseWriter, r *http.Re
return
}
- resp := refetchDuplicateResponse{RemovedPaths: make([]string, 0, len(res.Removed)), Blocked: res.Blocked}
- if resp.Blocked == nil {
- resp.Blocked = []string{}
- }
+ resp := refetchDuplicateResponse{RemovedPaths: make([]string, 0, len(res.Removed))}
if res.Kept != nil {
id := uuidToString(res.Kept.TrackID)
resp.KeptTrackID = &id
diff --git a/internal/api/admin_import.go b/internal/api/admin_import.go
deleted file mode 100644
index 207283ef..00000000
--- a/internal/api/admin_import.go
+++ /dev/null
@@ -1,285 +0,0 @@
-package api
-
-import (
- "encoding/json"
- "errors"
- "net/http"
-
- "github.com/jackc/pgx/v5"
-
- "git.fabledsword.com/bvandeusen/minstrel/internal/apierror"
- "git.fabledsword.com/bvandeusen/minstrel/internal/db/dbq"
- "git.fabledsword.com/bvandeusen/minstrel/internal/importer"
- "git.fabledsword.com/bvandeusen/minstrel/internal/lifecycle"
-)
-
-// Downloads and their imports: the Wanted view's Activity tab,
-// a person's "import anyway", "refuse" and "retry", and the Blocked tab.
-
-// downloadView is one grab and what its import made of it.
-type downloadView struct {
- ID string `json:"id"`
- WantID *string `json:"want_id"`
- WantTitle string `json:"want_title"`
- WantArtist string `json:"want_artist"`
- Name string `json:"name"`
- Indexer string `json:"indexer"`
- Client string `json:"client"`
- Quality string `json:"quality"`
- Origin string `json:"origin"`
- GrabbedAt string `json:"grabbed_at"`
- EndedAt *string `json:"ended_at"`
- EndReason string `json:"end_reason"`
- Progress float32 `json:"progress"`
- ImportState string `json:"import_state"`
- ImportReason string `json:"import_reason"`
- ImportedAt *string `json:"imported_at"`
- ImportedFiles int32 `json:"imported_files"`
- Destination string `json:"destination"`
- Edition string `json:"edition,omitempty"`
- Evidence *importer.Evidence `json:"evidence,omitempty"`
- Files []string `json:"files"`
- Skipped int `json:"skipped"`
- RehomedFrom string `json:"rehomed_from,omitempty"`
- // EndDetail says why an ended download ended.
- EndDetail string `json:"end_detail"`
- // Seeding: imported and still sharing; Ratio its share ratio as last
- // read; SeedReleasedAt when it was removed once it had shared enough.
- Seeding bool `json:"seeding"`
- Ratio float32 `json:"ratio"`
- SeedReleasedAt *string `json:"seed_released_at"`
- // Condition is what is wrong with it now, if anything.
- Condition *conditionView `json:"condition"`
-}
-
-// conditionView is an open row of the intervention ledger: what was seen,
-// since when, and how many times it was retried.
-type conditionView struct {
- Trigger string `json:"trigger"`
- Detail string `json:"detail"`
- Retries int32 `json:"retries"`
- Since string `json:"since"`
-}
-
-func downloadFrom(g dbq.ListGrabsRow) downloadView {
- v := downloadView{
- ID: uuidToString(g.ID), Name: g.Name, Indexer: g.Indexer, Client: g.Client, Quality: g.Quality,
- Origin: g.Origin, GrabbedAt: formatTimestamp(g.GrabbedAt), EndReason: g.EndReason,
- Progress: g.Progress, ImportState: g.ImportState, ImportReason: g.ImportReason,
- ImportedFiles: g.ImportedFiles, Destination: g.Destination, Files: []string{},
- EndDetail: g.EndDetail, Seeding: g.Seeding, Ratio: g.Ratio,
- }
- if g.SeedReleasedAt.Valid {
- at := formatTimestamp(g.SeedReleasedAt)
- v.SeedReleasedAt = &at
- }
- if g.Condition != nil {
- v.Condition = &conditionView{Trigger: *g.Condition, Detail: deref(g.ConditionDetail),
- Since: formatTimestamp(g.ConditionSince)}
- if g.ConditionRetries != nil {
- v.Condition.Retries = *g.ConditionRetries
- }
- }
- if g.WantID.Valid {
- id := uuidToString(g.WantID)
- v.WantID = &id
- }
- if g.WantTitle != nil {
- v.WantTitle = *g.WantTitle
- }
- if g.WantArtist != nil {
- v.WantArtist = *g.WantArtist
- }
- if g.EndedAt.Valid {
- at := formatTimestamp(g.EndedAt)
- v.EndedAt = &at
- }
- if g.ImportedAt.Valid {
- at := formatTimestamp(g.ImportedAt)
- v.ImportedAt = &at
- }
- var det importer.Detail
- if len(g.ImportDetail) > 0 && json.Unmarshal(g.ImportDetail, &det) == nil && det.Verdict.Outcome != "" {
- ev := det.Verdict.Evidence
- v.Evidence, v.Edition, v.Skipped, v.RehomedFrom = &ev, det.Edition, det.Skipped, det.RehomedFrom
- if det.Files != nil {
- v.Files = det.Files
- }
- }
- return v
-}
-
-type downloadsPage struct {
- Items []downloadView `json:"items"`
- Limit int `json:"limit"`
- Offset int `json:"offset"`
- // ClientsDown are the download clients not answering now.
- ClientsDown []clientDownView `json:"clients_down"`
-}
-
-type clientDownView struct {
- Client string `json:"client"`
- Detail string `json:"detail"`
- Since string `json:"since"`
-}
-
-// handleListDownloads implements GET /api/admin/downloads?offset=: running
-// downloads first, then the rest newest first.
-func (h *handlers) handleListDownloads(w http.ResponseWriter, r *http.Request) {
- limit, offset, err := parsePaging(r.URL.Query())
- if err != nil {
- writeErr(w, apierror.BadRequest("invalid_paging", "limit and offset must be numbers"))
- return
- }
- rows, err := dbq.New(h.pool).ListGrabs(r.Context(), dbq.ListGrabsParams{MaxRows: int32(limit), Skip: int32(offset)})
- if err != nil {
- writeErrWithLog(w, h.logger, "admin downloads: list", apierror.Internal(err))
- return
- }
- down, err := dbq.New(h.pool).ListClientsDown(r.Context())
- if err != nil {
- writeErrWithLog(w, h.logger, "admin downloads: clients down", apierror.Internal(err))
- return
- }
- out := downloadsPage{Items: make([]downloadView, 0, len(rows)), Limit: limit, Offset: offset,
- ClientsDown: make([]clientDownView, 0, len(down))}
- for _, g := range rows {
- out.Items = append(out.Items, downloadFrom(g))
- }
- for _, c := range down {
- out.ClientsDown = append(out.ClientsDown, clientDownView{Client: c.Client, Detail: c.Detail,
- Since: formatTimestamp(c.FirstSeen)})
- }
- writeJSON(w, http.StatusOK, out)
-}
-
-// handleAbandonDownload implements POST /api/admin/downloads/{id}/abandon: a
-// running download is given up, its release set aside, the download removed
-// and the album searched again.
-func (h *handlers) handleAbandonDownload(w http.ResponseWriter, r *http.Request) {
- admin, ok := requireUser(w, r)
- if !ok {
- return
- }
- id, ok := requireURLUUID(w, r, "id")
- if !ok {
- return
- }
- if _, err := h.lifecycle.Abandon(r.Context(), id, admin.ID); err != nil {
- switch {
- case errors.Is(err, pgx.ErrNoRows), errors.Is(err, lifecycle.ErrNotAbandonable):
- writeErr(w, apierror.Conflict("not_running", "that download is no longer running"))
- default:
- writeErrWithLog(w, h.logger, "admin downloads: abandon", apierror.Internal(err))
- }
- return
- }
- w.WriteHeader(http.StatusNoContent)
-}
-
-// handleImportAnyway implements POST /api/admin/downloads/{id}/import: a
-// download the check held for a person (or one that failed) is imported on
-// the next pass whatever the check says.
-func (h *handlers) handleImportAnyway(w http.ResponseWriter, r *http.Request) {
- id, ok := requireURLUUID(w, r, "id")
- if !ok {
- return
- }
- if _, err := dbq.New(h.pool).ForceGrabImport(r.Context(), id); err != nil {
- h.writeImportErr(w, "import anyway", err)
- return
- }
- w.WriteHeader(http.StatusNoContent)
-}
-
-// handleRetryImport implements POST /api/admin/downloads/{id}/retry.
-func (h *handlers) handleRetryImport(w http.ResponseWriter, r *http.Request) {
- id, ok := requireURLUUID(w, r, "id")
- if !ok {
- return
- }
- if _, err := dbq.New(h.pool).RetryGrabImport(r.Context(), id); err != nil {
- h.writeImportErr(w, "retry", err)
- return
- }
- w.WriteHeader(http.StatusNoContent)
-}
-
-// handleRefuseDownload implements POST /api/admin/downloads/{id}/refuse: the
-// release is blocked, the download removed and the album searched again.
-func (h *handlers) handleRefuseDownload(w http.ResponseWriter, r *http.Request) {
- admin, ok := requireUser(w, r)
- if !ok {
- return
- }
- id, ok := requireURLUUID(w, r, "id")
- if !ok {
- return
- }
- if _, err := h.importer.Refuse(r.Context(), id, admin.ID); err != nil {
- h.writeImportErr(w, "refuse", err)
- return
- }
- w.WriteHeader(http.StatusNoContent)
-}
-
-func (h *handlers) writeImportErr(w http.ResponseWriter, what string, err error) {
- switch {
- case errors.Is(err, pgx.ErrNoRows), errors.Is(err, importer.ErrNotDecidable):
- writeErr(w, apierror.Conflict("not_decidable", "that download is not waiting on a decision"))
- default:
- writeErrWithLog(w, h.logger, "admin downloads: "+what, apierror.Internal(err))
- }
-}
-
-type releaseBlockView struct {
- InfoHash string `json:"info_hash"`
- Name string `json:"name"`
- Reason string `json:"reason"`
- Detail string `json:"detail"`
- BlockedAt string `json:"blocked_at"`
- // ExpiresAt is when a release given up on may be taken again; null is
- // for good.
- ExpiresAt *string `json:"expires_at"`
- WantTitle string `json:"want_title"`
- WantArtist string `json:"want_artist"`
-}
-
-type releaseBlocksPage struct {
- Items []releaseBlockView `json:"items"`
- Limit int `json:"limit"`
- Offset int `json:"offset"`
-}
-
-// handleListReleaseBlocks implements GET /api/admin/release-blocks?offset=:
-// every release set aside, by a person or by a check, newest first.
-func (h *handlers) handleListReleaseBlocks(w http.ResponseWriter, r *http.Request) {
- limit, offset, err := parsePaging(r.URL.Query())
- if err != nil {
- writeErr(w, apierror.BadRequest("invalid_paging", "limit and offset must be numbers"))
- return
- }
- rows, err := dbq.New(h.pool).ListReleaseBlocksPage(r.Context(), dbq.ListReleaseBlocksPageParams{
- MaxRows: int32(limit), Skip: int32(offset)})
- if err != nil {
- writeErrWithLog(w, h.logger, "admin release blocks: list", apierror.Internal(err))
- return
- }
- out := releaseBlocksPage{Items: make([]releaseBlockView, 0, len(rows)), Limit: limit, Offset: offset}
- for _, b := range rows {
- v := releaseBlockView{InfoHash: b.InfoHash, Name: b.Name, Reason: b.Reason, Detail: b.Detail,
- BlockedAt: formatTimestamp(b.BlockedAt)}
- if b.ExpiresAt.Valid {
- at := formatTimestamp(b.ExpiresAt)
- v.ExpiresAt = &at
- }
- if b.WantTitle != nil {
- v.WantTitle = *b.WantTitle
- }
- if b.WantArtist != nil {
- v.WantArtist = *b.WantArtist
- }
- out.Items = append(out.Items, v)
- }
- writeJSON(w, http.StatusOK, out)
-}
diff --git a/internal/api/admin_library_missing.go b/internal/api/admin_library_missing.go
index 30a3a4a8..59fd5ff4 100644
--- a/internal/api/admin_library_missing.go
+++ b/internal/api/admin_library_missing.go
@@ -2,9 +2,6 @@ package api
import (
"net/http"
- "time"
-
- "github.com/jackc/pgx/v5/pgtype"
"git.fabledsword.com/bvandeusen/minstrel/internal/db/dbq"
)
@@ -15,7 +12,7 @@ import (
// the point: the recording is still a known thing with a title, an artist and
// a play history — only its bytes are absent. LastPlayedAt is nullable because
// plenty of missing files were never played, and that is exactly the signal an
-// operator wants when deciding whether to bother re-acquiring one.
+// operator wants when deciding whether a lost file is worth replacing.
type missingTrackView struct {
TrackID string `json:"track_id"`
Title string `json:"title"`
@@ -29,18 +26,6 @@ type missingTrackView struct {
LastPlayedAt *string `json:"last_played_at"`
}
-// reacquisitionStateView is what the wanted list says about one album: when
-// it was wanted back, and when the library
-// held it again. A missing album with no want yet is still inside the grace
-// window, or MusicBrainz cannot name it. Want is that want as the wanted list
-// sends it, so the Missing page can show its search and download and act on
-// it.
-type reacquisitionStateView struct {
- WantedAt *string `json:"wanted_at"`
- HeldAt *string `json:"held_at"`
- Want *wantView `json:"want"`
-}
-
// missingGroupView is a directory's worth of missing tracks.
//
// Grouping is the whole ergonomic argument for this surface: a reorganised
@@ -52,10 +37,6 @@ type missingGroupView struct {
Directory string `json:"directory"`
MissingSince string `json:"missing_since"`
Tracks []missingTrackView `json:"tracks"`
- // Nil when nothing has been attempted for this group's album — the
- // common case for a folder that just went missing, and distinct from
- // an attempts:0 record, which cannot occur.
- Reacquisition *reacquisitionStateView `json:"reacquisition"`
}
// adminMissingResponse is the paged envelope. Total counts TRACKS, not
@@ -99,7 +80,6 @@ func (h *handlers) handleListMissingTracks(w http.ResponseWriter, r *http.Reques
}
groups := groupMissingByDirectory(rows)
- h.attachReacquisitionState(r, q, rows, groups)
out := adminMissingResponse{
Total: total,
@@ -154,109 +134,3 @@ func groupMissingByDirectory(rows []dbq.ListMissingTracksRow) []missingGroupView
}
return groups
}
-
-// attachReacquisitionState decorates each group with its album's want.
-//
-// Best-effort: this is context on a list whose primary job is showing what is
-// missing, so a failure here leaves the groups bare rather than failing the
-// page. One batched query for the whole page, not one per group.
-//
-// A group is keyed by directory while a want is keyed by album, and those
-// line up in practice (an album's files live in one folder) but are not
-// guaranteed to — a directory holding two albums takes the first album's
-// state, which is the same album its first track belongs to.
-func (h *handlers) attachReacquisitionState(
- r *http.Request,
- q *dbq.Queries,
- rows []dbq.ListMissingTracksRow,
- groups []missingGroupView,
-) {
- if len(groups) == 0 {
- return
- }
- // Directory -> the album its first row belongs to, matching the order
- // groupMissingByDirectory folded them in.
- dirAlbum := make(map[string]pgtype.UUID, len(groups))
- ids := make([]pgtype.UUID, 0, len(groups))
- // The latest loss in each folder: a want held before it is the album
- // as it was before its files went, not the library holding it again.
- dirLost := make(map[string]time.Time, len(groups))
- for _, row := range rows {
- if row.MissingSince.Valid && row.MissingSince.Time.After(dirLost[row.Directory]) {
- dirLost[row.Directory] = row.MissingSince.Time
- }
- if _, seen := dirAlbum[row.Directory]; seen {
- continue
- }
- dirAlbum[row.Directory] = row.AlbumID
- ids = append(ids, row.AlbumID)
- }
-
- wants, err := q.GetWantsForAlbums(r.Context(), ids)
- if err != nil {
- h.logger.Warn("admin missing: want lookup failed", "err", err)
- return
- }
- // Newest want first, so the first seen per album is its current one.
- byAlbum := make(map[string]dbq.GetWantsForAlbumsRow, len(wants))
- for _, w := range wants {
- if _, seen := byAlbum[uuidToString(w.AlbumID)]; !seen {
- byAlbum[uuidToString(w.AlbumID)] = w
- }
- }
- full := h.wantViewsByID(r, q, byAlbum)
- for i := range groups {
- albumID, ok := dirAlbum[groups[i].Directory]
- if !ok {
- continue
- }
- w, ok := byAlbum[uuidToString(albumID)]
- if !ok {
- continue
- }
- // Held before its files went (every library album is on the list,
- // held): not wanted back yet, which the sweep does once
- // the grace window has passed.
- if w.FulfilledAt.Valid && !w.FulfilledAt.Time.After(dirLost[groups[i].Directory]) {
- continue
- }
- view := &reacquisitionStateView{}
- wantedAt := formatTimestamp(w.WantedAt)
- view.WantedAt = &wantedAt
- if w.FulfilledAt.Valid {
- heldAt := formatTimestamp(w.FulfilledAt)
- view.HeldAt = &heldAt
- }
- if v, ok := full[uuidToString(w.WantID)]; ok {
- view.Want = &v
- }
- groups[i].Reacquisition = view
- }
-}
-
-// wantViewsByID reads each album's current want in full, keyed by want id.
-// Best-effort like the state it decorates: a failure leaves the groups with
-// their dates and no want.
-func (h *handlers) wantViewsByID(
- r *http.Request,
- q *dbq.Queries,
- byAlbum map[string]dbq.GetWantsForAlbumsRow,
-) map[string]wantView {
- ids := make([]pgtype.UUID, 0, len(byAlbum))
- for _, w := range byAlbum {
- ids = append(ids, w.WantID)
- }
- out := make(map[string]wantView, len(ids))
- if len(ids) == 0 {
- return out
- }
- rows, err := q.ListWants(r.Context(), dbq.ListWantsParams{State: "all", WantIds: ids, MaxRows: int32(len(ids))})
- if err != nil {
- h.logger.Warn("admin missing: want detail lookup failed", "err", err)
- return out
- }
- for _, row := range rows {
- out[uuidToString(row.ID)] = wantViewOf(row)
- }
- return out
-}
diff --git a/internal/api/admin_library_missing_wants_test.go b/internal/api/admin_library_missing_wants_test.go
deleted file mode 100644
index 6710580d..00000000
--- a/internal/api/admin_library_missing_wants_test.go
+++ /dev/null
@@ -1,84 +0,0 @@
-package api
-
-import (
- "context"
- "encoding/json"
- "net/http"
- "net/http/httptest"
- "testing"
-
- "github.com/jackc/pgx/v5/pgtype"
-)
-
-// seedMissingWant marks a track's file missing for two days and puts its
-// album on the wanted list, held at heldDaysAgo (0 for an open want) and
-// last searched with nothing found.
-func seedMissingWant(t *testing.T, h *handlers, trackID, albumID pgtype.UUID, group string, heldDaysAgo int) string {
- t.Helper()
- ctx := context.Background()
- if _, err := h.pool.Exec(ctx,
- `UPDATE tracks SET missing_since = now() - interval '2 days' WHERE id = $1`, trackID); err != nil {
- t.Fatalf("mark missing: %v", err)
- }
- if _, err := h.pool.Exec(ctx, `DELETE FROM wants WHERE release_group_mbid = $1`, group); err != nil {
- t.Fatalf("clear want: %v", err)
- }
- var id pgtype.UUID
- if err := h.pool.QueryRow(ctx,
- `INSERT INTO wants (release_group_mbid, artist_mbid, artist_name, title, reason, album_id,
- wanted_at, fulfilled_at, searched_at, search_outcome)
- VALUES ($1, 'artist-'||$1, 'Artist', 'Album '||$1, 'library', $2,
- now() - interval '30 days',
- CASE WHEN $3::int > 0 THEN now() - make_interval(days => $3::int) END,
- now() - interval '1 hour', 'nothing')
- RETURNING id`, group, albumID, heldDaysAgo).Scan(&id); err != nil {
- t.Fatalf("insert want: %v", err)
- }
- return uuidToString(id)
-}
-
-// A folder's album that was held before its files went is not wanted back
-// yet (every library album is on the list, held); an open want comes with
-// its search, so the Missing page can show it and act on it.
-func TestListMissingTracks_WantState(t *testing.T) {
- h, pool := testHandlers(t)
- truncateLibrary(t, pool)
- admin := seedUser(t, pool, "admin", "pw", true)
-
- heldTrack, _ := seedTrackOnDisk(t, h, "missheld")
- openTrack, _ := seedTrackOnDisk(t, h, "missopen")
- seedMissingWant(t, h, heldTrack.ID, heldTrack.AlbumID, "rg-missing-held", 10)
- openID := seedMissingWant(t, h, openTrack.ID, openTrack.AlbumID, "rg-missing-open", 0)
-
- req := withUser(httptest.NewRequest(http.MethodGet, "/api/admin/library/missing", nil), admin)
- w := httptest.NewRecorder()
- h.handleListMissingTracks(w, req)
- if w.Code != http.StatusOK {
- t.Fatalf("status %d: %s", w.Code, w.Body.String())
- }
- var resp adminMissingResponse
- if err := json.Unmarshal(w.Body.Bytes(), &resp); err != nil {
- t.Fatalf("decode: %v", err)
- }
- if len(resp.Groups) != 2 {
- t.Fatalf("groups = %d, want 2", len(resp.Groups))
- }
- byTitle := map[string]missingGroupView{}
- for _, g := range resp.Groups {
- byTitle[g.Tracks[0].Title] = g
- }
-
- if r := byTitle["AdmQ Track missheld"].Reacquisition; r != nil {
- t.Errorf("held before the loss: reacquisition = %+v, want none", r)
- }
- r := byTitle["AdmQ Track missopen"].Reacquisition
- if r == nil || r.WantedAt == nil || r.HeldAt != nil {
- t.Fatalf("open want: reacquisition = %+v, want wanted and not held", r)
- }
- if r.Want == nil || r.Want.ID != openID {
- t.Fatalf("open want: want = %+v, want id %s", r.Want, openID)
- }
- if r.Want.Search == nil || r.Want.Search.Outcome != "nothing" {
- t.Errorf("open want: search = %+v, want outcome nothing", r.Want.Search)
- }
-}
diff --git a/internal/api/admin_reacquisition.go b/internal/api/admin_reacquisition.go
deleted file mode 100644
index 01d841c1..00000000
--- a/internal/api/admin_reacquisition.go
+++ /dev/null
@@ -1,78 +0,0 @@
-package api
-
-import (
- "encoding/json"
- "errors"
- "net/http"
-
- "git.fabledsword.com/bvandeusen/minstrel/internal/apierror"
- "git.fabledsword.com/bvandeusen/minstrel/internal/db/dbq"
- "git.fabledsword.com/bvandeusen/minstrel/internal/reacquisition"
-)
-
-// reacquisitionSettingsResp is the admin card's payload.
-//
-// Carries more than the stored settings: [UnnameableAlbums] is the count of
-// albums with missing files that can never be wanted automatically because
-// neither they nor their artist has an MBID. The card states that number
-// rather than leaving an admin to wonder why some rows never get wanted —
-// a feature that silently does nothing for part of its input reads as broken.
-type reacquisitionSettingsResp struct {
- Enabled bool `json:"enabled"`
- GraceHours int32 `json:"grace_hours"`
- MaxPerPass int32 `json:"max_per_pass"`
-
- UnnameableAlbums int64 `json:"unnameable_albums"`
-}
-
-// handleGetReacquisitionSettings implements GET
-// /api/admin/library/reacquisition.
-func (h *handlers) handleGetReacquisitionSettings(w http.ResponseWriter, r *http.Request) {
- writeJSON(w, http.StatusOK, h.reacquisitionPayload(r))
-}
-
-// handleUpdateReacquisitionSettings implements PUT
-// /api/admin/library/reacquisition.
-func (h *handlers) handleUpdateReacquisitionSettings(w http.ResponseWriter, r *http.Request) {
- var req reacquisitionSettingsResp
- if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
- writeErr(w, apierror.BadRequest("invalid_body", "malformed JSON"))
- return
- }
- _, err := h.reacqSettings.Set(r.Context(), reacquisition.Settings{
- Enabled: req.Enabled,
- GraceHours: req.GraceHours,
- MaxPerPass: req.MaxPerPass,
- })
- if err != nil {
- // The Go-side validation mirrors migration 0056's CHECKs so the
- // operator gets a readable message naming the field, rather than a
- // constraint-violation string leaking through as a 500.
- if errors.Is(err, reacquisition.ErrOutOfRange) {
- writeErr(w, apierror.BadRequest("invalid_setting", err.Error()))
- return
- }
- writeErrWithLog(w, h.logger, "admin reacquisition: update failed", apierror.Internal(err))
- return
- }
- // Echo the payload recomputed under the new values so the card reflects
- // what it just did without a reload.
- writeJSON(w, http.StatusOK, h.reacquisitionPayload(r))
-}
-
-func (h *handlers) reacquisitionPayload(r *http.Request) reacquisitionSettingsResp {
- cur := h.reacqSettings.Get()
- out := reacquisitionSettingsResp{
- Enabled: cur.Enabled,
- GraceHours: cur.GraceHours,
- MaxPerPass: cur.MaxPerPass,
- }
- // Best-effort: the settings are the point of this endpoint, and failing
- // the whole card because a count query hiccuped would be the wrong trade.
- if n, err := dbq.New(h.pool).CountAlbumsMissingWithoutMbid(r.Context()); err == nil {
- out.UnnameableAlbums = n
- } else {
- h.logger.Warn("admin reacquisition: unnameable count failed", "err", err)
- }
- return out
-}
diff --git a/internal/api/admin_recommendation_tuning.go b/internal/api/admin_recommendation_tuning.go
index d5e69799..4f77ae02 100644
--- a/internal/api/admin_recommendation_tuning.go
+++ b/internal/api/admin_recommendation_tuning.go
@@ -67,30 +67,15 @@ func tasteRespFrom(t recsettings.TasteTuning) tasteTuningResp {
}
}
-// discoverTuningResp is the Discover scope on the wire.
-type discoverTuningResp struct {
- TagOverlapWeight float64 `json:"tag_overlap_weight"`
- SnoozeDays float64 `json:"snooze_days"`
-}
-
-func discoverRespFrom(d recsettings.DiscoverTuning) discoverTuningResp {
- return discoverTuningResp{
- TagOverlapWeight: d.TagOverlapWeight,
- SnoozeDays: d.SnoozeDays,
- }
-}
-
// tuningSnapshot is both the GET response and the post-mutation echo:
// current values alongside shipped defaults so the card can mark
// which knobs deviate.
type tuningSnapshot struct {
Profiles map[string]weightsResp `json:"profiles"`
Taste tasteTuningResp `json:"taste"`
- Discover discoverTuningResp `json:"discover"`
Shipped struct {
Profiles map[string]weightsResp `json:"profiles"`
Taste tasteTuningResp `json:"taste"`
- Discover discoverTuningResp `json:"discover"`
} `json:"shipped"`
}
@@ -102,14 +87,12 @@ func (h *handlers) tuningSnapshot() tuningSnapshot {
recsettings.ScopeSongsLike: weightsRespFrom(h.recSettings.Weights(recsettings.ScopeSongsLike)),
}
out.Taste = tasteRespFrom(h.recSettings.Taste())
- out.Discover = discoverRespFrom(h.recSettings.Discover())
out.Shipped.Profiles = map[string]weightsResp{
recsettings.ScopeRadio: weightsRespFrom(recsettings.ShippedRadioWeights()),
recsettings.ScopeDailyMix: weightsRespFrom(recsettings.ShippedDailyMixWeights()),
recsettings.ScopeSongsLike: weightsRespFrom(recsettings.ShippedSongsLikeWeights()),
}
out.Shipped.Taste = tasteRespFrom(recsettings.ShippedTasteTuning())
- out.Shipped.Discover = discoverRespFrom(recsettings.ShippedDiscoverTuning())
return out
}
@@ -143,8 +126,6 @@ func (h *handlers) handlePatchRecommendationTuning(w http.ResponseWriter, r *htt
switch scope {
case recsettings.ScopeTaste:
err = h.recSettings.UpdateTaste(r.Context(), body.Values)
- case recsettings.ScopeDiscover:
- err = h.recSettings.UpdateDiscover(r.Context(), body.Values)
default:
err = h.recSettings.UpdateProfile(r.Context(), scope, body.Values)
}
diff --git a/internal/api/admin_requests.go b/internal/api/admin_requests.go
deleted file mode 100644
index 4fd395eb..00000000
--- a/internal/api/admin_requests.go
+++ /dev/null
@@ -1,190 +0,0 @@
-package api
-
-import (
- "context"
- "encoding/json"
- "errors"
- "net/http"
- "strconv"
-
- "github.com/go-chi/chi/v5"
- "github.com/jackc/pgx/v5/pgtype"
-
- "git.fabledsword.com/bvandeusen/minstrel/internal/auth"
- "git.fabledsword.com/bvandeusen/minstrel/internal/db/dbq"
- "git.fabledsword.com/bvandeusen/minstrel/internal/notifications"
- "git.fabledsword.com/bvandeusen/minstrel/internal/requests"
- "git.fabledsword.com/bvandeusen/minstrel/internal/wanted"
-)
-
-// validRequestStatuses is the set of allowed values for the ?status= param.
-var validRequestStatuses = map[string]bool{
- "pending": true,
- "approved": true,
- "rejected": true,
- "completed": true,
- "failed": true,
-}
-
-// handleListAdminRequests implements GET /api/admin/requests.
-// ?status= defaults to "pending"; ?limit= defaults to 50, max 200.
-func (h *handlers) handleListAdminRequests(w http.ResponseWriter, r *http.Request) {
- status := r.URL.Query().Get("status")
- if status == "" {
- status = "pending"
- }
- if !validRequestStatuses[status] {
- writeAdminJSONErr(w, http.StatusBadRequest, "invalid_status")
- return
- }
-
- limitStr := r.URL.Query().Get("limit")
- limit := 50
- if limitStr != "" {
- v, err := strconv.Atoi(limitStr)
- if err != nil || v <= 0 {
- writeAdminJSONErr(w, http.StatusBadRequest, "invalid_limit")
- return
- }
- if v > 200 {
- v = 200
- }
- limit = v
- }
-
- rows, err := h.requests.ListByStatus(r.Context(), status, int32(limit))
- if err != nil {
- h.logger.Error("admin: list requests", "err", err)
- writeAdminJSONErr(w, http.StatusInternalServerError, "server_error")
- return
- }
-
- q := dbq.New(h.pool)
- names := requesterNames(r.Context(), q, rows)
- out := make([]requestView, 0, len(rows))
- for _, row := range rows {
- v := requestViewFrom(row)
- v.Username = names[row.UserID]
- fillProgress(r.Context(), q, row, &v)
- out = append(out, v)
- }
- writeJSON(w, http.StatusOK, out)
-}
-
-// requesterNames maps each request's user id to that user's username, in one
-// query for the whole page. Like fillProgress, a failed lookup is swallowed:
-// the names are a reporting nicety, and the page falls back to the id.
-func requesterNames(ctx context.Context, q *dbq.Queries, rows []dbq.Request) map[pgtype.UUID]string {
- ids := make([]pgtype.UUID, 0, len(rows))
- for _, row := range rows {
- ids = append(ids, row.UserID)
- }
- names := make(map[pgtype.UUID]string, len(rows))
- users, err := q.ListUsernamesByIDs(ctx, ids)
- if err != nil {
- return names
- }
- for _, u := range users {
- names[u.ID] = u.Username
- }
- return names
-}
-
-// approveRequestBody is the optional JSON body for POST
-// /api/admin/requests/:id/approve. profile_id picks the release-type profile an
-// artist request is followed on; omitted, the default profile.
-type approveRequestBody struct {
- ProfileID string `json:"profile_id"`
-}
-
-// handleApproveRequest implements POST /api/admin/requests/:id/approve. The
-// request goes on the wanted list: an album or track request
-// wants its album, an artist request follows the artist.
-func (h *handlers) handleApproveRequest(w http.ResponseWriter, r *http.Request) {
- admin, ok := auth.UserFromContext(r.Context())
- if !ok {
- writeAdminJSONErr(w, http.StatusUnauthorized, "unauthorized")
- return
- }
-
- id, ok := parseUUID(chi.URLParam(r, "id"))
- if !ok {
- writeAdminJSONErr(w, http.StatusBadRequest, "invalid_id")
- return
- }
-
- var body approveRequestBody
- // Body is entirely optional; ignore decode errors.
- _ = json.NewDecoder(r.Body).Decode(&body)
- var profile pgtype.UUID
- if body.ProfileID != "" {
- if profile, ok = parseUUID(body.ProfileID); !ok {
- writeAdminJSONErr(w, http.StatusBadRequest, "invalid_profile")
- return
- }
- }
-
- row, err := h.requests.Approve(r.Context(), id, admin.ID, profile)
- if err != nil {
- switch {
- case errors.Is(err, requests.ErrNotFound):
- writeAdminJSONErr(w, http.StatusNotFound, "request_not_found")
- case errors.Is(err, requests.ErrNotPending):
- writeAdminJSONErr(w, http.StatusConflict, "request_not_pending")
- case errors.Is(err, wanted.ErrProfileNotFound):
- writeAdminJSONErr(w, http.StatusBadRequest, "profile_not_found")
- case errors.Is(err, wanted.ErrNotFollowable):
- writeAdminJSONErr(w, http.StatusBadRequest, "not_followable")
- default:
- h.logger.Error("admin: approve request", "err", err)
- writeAdminJSONErr(w, http.StatusInternalServerError, "server_error")
- }
- return
- }
-
- h.notifyRequestDecided(r.Context(), notifications.KindRequestApproved, admin, row, "")
- h.publishRequestStatusChanged(row)
- writeJSON(w, http.StatusOK, requestViewFrom(row))
-}
-
-// rejectRequestBody is the optional JSON body for POST
-// /api/admin/requests/:id/reject.
-type rejectRequestBody struct {
- Notes string `json:"notes"`
-}
-
-// handleRejectRequest implements POST /api/admin/requests/:id/reject.
-func (h *handlers) handleRejectRequest(w http.ResponseWriter, r *http.Request) {
- admin, ok := auth.UserFromContext(r.Context())
- if !ok {
- writeAdminJSONErr(w, http.StatusUnauthorized, "unauthorized")
- return
- }
-
- id, ok := parseUUID(chi.URLParam(r, "id"))
- if !ok {
- writeAdminJSONErr(w, http.StatusBadRequest, "invalid_id")
- return
- }
-
- var body rejectRequestBody
- _ = json.NewDecoder(r.Body).Decode(&body)
-
- row, err := h.requests.Reject(r.Context(), id, admin.ID, body.Notes)
- if err != nil {
- switch {
- case errors.Is(err, requests.ErrNotFound):
- writeAdminJSONErr(w, http.StatusNotFound, "request_not_found")
- case errors.Is(err, requests.ErrNotPending):
- writeAdminJSONErr(w, http.StatusConflict, "request_not_pending")
- default:
- h.logger.Error("admin: reject request", "err", err)
- writeAdminJSONErr(w, http.StatusInternalServerError, "server_error")
- }
- return
- }
-
- h.notifyRequestDecided(r.Context(), notifications.KindRequestRejected, admin, row, body.Notes)
- h.publishRequestStatusChanged(row)
- writeJSON(w, http.StatusOK, requestViewFrom(row))
-}
diff --git a/internal/api/admin_requests_test.go b/internal/api/admin_requests_test.go
deleted file mode 100644
index 734eadfc..00000000
--- a/internal/api/admin_requests_test.go
+++ /dev/null
@@ -1,335 +0,0 @@
-package api
-
-import (
- "bytes"
- "context"
- "encoding/json"
- "net/http"
- "net/http/httptest"
- "testing"
-
- "github.com/go-chi/chi/v5"
-
- "git.fabledsword.com/bvandeusen/minstrel/internal/auth"
- "git.fabledsword.com/bvandeusen/minstrel/internal/db/dbq"
-)
-
-// newAdminRequestsRouter builds a test chi router with the three admin/requests
-// endpoints. RequireAdmin middleware is applied. Tests inject a user into
-// context directly (skipping RequireUser).
-func newAdminRequestsRouter(h *handlers) chi.Router {
- r := chi.NewRouter()
- r.Route("/api/admin", func(admin chi.Router) {
- admin.Use(auth.RequireAdmin())
- admin.Get("/requests", h.handleListAdminRequests)
- admin.Post("/requests/{id}/approve", h.handleApproveRequest)
- admin.Post("/requests/{id}/reject", h.handleRejectRequest)
- })
- return r
-}
-
-// doAdminRequestReq fires an HTTP request at the admin requests router with
-// the given user injected into context (bypassing RequireUser).
-func doAdminRequestReq(t *testing.T, h *handlers, method, path string, body []byte, user dbq.User) *httptest.ResponseRecorder {
- t.Helper()
- var reqBody *bytes.Buffer
- if body != nil {
- reqBody = bytes.NewBuffer(body)
- } else {
- reqBody = bytes.NewBuffer(nil)
- }
- req := httptest.NewRequest(method, path, reqBody)
- if body != nil {
- req.Header.Set("Content-Type", "application/json")
- }
- req = withUser(req, user)
- w := httptest.NewRecorder()
- newAdminRequestsRouter(h).ServeHTTP(w, req)
- return w
-}
-
-// seedPendingArtistRequest creates a pending artist-kind request row
-// for the given user and returns its parsed requestView.
-func seedPendingArtistRequest(t *testing.T, h *handlers, user dbq.User, artistMBID, artistName string) requestView {
- t.Helper()
- return createArtistRequest(t, h, user, artistMBID, artistName)
-}
-
-// TestHandleListAdminRequests_DefaultStatus_Pending seeds 2 pending + 1
-// rejected and verifies that GET without ?status returns only the 2 pending
-// rows.
-func TestHandleListAdminRequests_DefaultStatus_Pending(t *testing.T) {
- h, _ := testHandlers(t)
- resetRequests(t, h)
-
- alice := seedUser(t, h.pool, "alice", "pw", false)
- admin := seedUser(t, h.pool, "admin", "pw", true)
-
- createArtistRequest(t, h, alice, "mbid-pending-1", "Pending Artist 1")
- createArtistRequest(t, h, alice, "mbid-pending-2", "Pending Artist 2")
-
- // Reject one request directly via service.
- rv3 := createArtistRequest(t, h, alice, "mbid-reject-1", "Reject Me")
- _, err := h.requests.Reject(context.Background(), rv3.ID, admin.ID, "test")
- if err != nil {
- t.Fatalf("reject seed: %v", err)
- }
-
- w := doAdminRequestReq(t, h, http.MethodGet, "/api/admin/requests", nil, admin)
- if w.Code != http.StatusOK {
- t.Fatalf("status = %d, want 200; body = %s", w.Code, w.Body.String())
- }
-
- var results []requestView
- if err := json.Unmarshal(w.Body.Bytes(), &results); err != nil {
- t.Fatalf("decode: %v; body = %s", err, w.Body.String())
- }
- if len(results) != 2 {
- t.Fatalf("len(results) = %d, want 2 (pending only)", len(results))
- }
- for _, rv := range results {
- if rv.Status != "pending" {
- t.Errorf("result status = %q, want pending", rv.Status)
- }
- if rv.Username != alice.Username {
- t.Errorf("result username = %q, want %q (the requester)", rv.Username, alice.Username)
- }
- }
-}
-
-// TestHandleListAdminRequests_StatusFilter verifies GET ?status=rejected
-// returns only rejected rows.
-func TestHandleListAdminRequests_StatusFilter(t *testing.T) {
- h, _ := testHandlers(t)
- resetRequests(t, h)
-
- alice := seedUser(t, h.pool, "alice", "pw", false)
- admin := seedUser(t, h.pool, "admin", "pw", true)
-
- createArtistRequest(t, h, alice, "mbid-pending-f1", "Pending Filter 1")
-
- rv2 := createArtistRequest(t, h, alice, "mbid-rejected-f1", "Rejected Filter 1")
- _, err := h.requests.Reject(context.Background(), rv2.ID, admin.ID, "")
- if err != nil {
- t.Fatalf("reject seed: %v", err)
- }
-
- w := doAdminRequestReq(t, h, http.MethodGet, "/api/admin/requests?status=rejected", nil, admin)
- if w.Code != http.StatusOK {
- t.Fatalf("status = %d, want 200; body = %s", w.Code, w.Body.String())
- }
-
- var results []requestView
- if err := json.Unmarshal(w.Body.Bytes(), &results); err != nil {
- t.Fatalf("decode: %v; body = %s", err, w.Body.String())
- }
- if len(results) != 1 {
- t.Fatalf("len(results) = %d, want 1 rejected row", len(results))
- }
- if results[0].Status != "rejected" {
- t.Errorf("status = %q, want rejected", results[0].Status)
- }
-}
-
-// TestHandleListAdminRequests_BadStatus_400 verifies GET with an invalid
-// ?status returns 400.
-func TestHandleListAdminRequests_BadStatus_400(t *testing.T) {
- h, _ := testHandlers(t)
- resetRequests(t, h)
-
- admin := seedUser(t, h.pool, "admin", "pw", true)
-
- w := doAdminRequestReq(t, h, http.MethodGet, "/api/admin/requests?status=garbage", nil, admin)
- if w.Code != http.StatusBadRequest {
- t.Fatalf("status = %d, want 400; body = %s", w.Code, w.Body.String())
- }
- var resp map[string]string
- if err := json.Unmarshal(w.Body.Bytes(), &resp); err != nil {
- t.Fatalf("decode: %v; body = %s", err, w.Body.String())
- }
- if resp["error"] != "invalid_status" {
- t.Errorf("error = %q, want invalid_status", resp["error"])
- }
-}
-
-// TestHandleApproveRequest_NotPending_409 pre-rejects a row and verifies
-// POST /approve → 409 request_not_pending.
-func TestHandleApproveRequest_NotPending_409(t *testing.T) {
- h, _ := testHandlers(t)
- resetRequests(t, h)
-
- alice := seedUser(t, h.pool, "alice", "pw", false)
- admin := seedUser(t, h.pool, "admin", "pw", true)
-
- rv := seedPendingArtistRequest(t, h, alice, "artist-mbid-notpending", "Not Pending")
-
- // Pre-reject the row.
- _, err := h.requests.Reject(context.Background(), rv.ID, admin.ID, "pre-rejected")
- if err != nil {
- t.Fatalf("pre-reject: %v", err)
- }
-
- w := doAdminRequestReq(t, h, http.MethodPost, "/api/admin/requests/"+uuidToString(rv.ID)+"/approve", nil, admin)
- if w.Code != http.StatusConflict {
- t.Fatalf("status = %d, want 409; body = %s", w.Code, w.Body.String())
- }
- var resp map[string]string
- if err := json.Unmarshal(w.Body.Bytes(), &resp); err != nil {
- t.Fatalf("decode: %v; body = %s", err, w.Body.String())
- }
- if resp["error"] != "request_not_pending" {
- t.Errorf("error = %q, want request_not_pending", resp["error"])
- }
-}
-
-// TestHandleRejectRequest_HappyPath verifies POST /reject with notes returns
-// 200 and the row transitions to rejected with notes saved.
-func TestHandleRejectRequest_HappyPath(t *testing.T) {
- h, _ := testHandlers(t)
- resetRequests(t, h)
-
- alice := seedUser(t, h.pool, "alice", "pw", false)
- admin := seedUser(t, h.pool, "admin", "pw", true)
-
- rv := seedPendingArtistRequest(t, h, alice, "artist-mbid-reject", "Reject Me")
-
- body := []byte(`{"notes":"low-quality release; will look for the remaster"}`)
- w := doAdminRequestReq(t, h, http.MethodPost, "/api/admin/requests/"+uuidToString(rv.ID)+"/reject", body, admin)
- if w.Code != http.StatusOK {
- t.Fatalf("status = %d, want 200; body = %s", w.Code, w.Body.String())
- }
-
- var got requestView
- if err := json.Unmarshal(w.Body.Bytes(), &got); err != nil {
- t.Fatalf("decode: %v; body = %s", err, w.Body.String())
- }
- if got.Status != "rejected" {
- t.Errorf("status = %q, want rejected", got.Status)
- }
- if got.Notes == nil || *got.Notes != "low-quality release; will look for the remaster" {
- t.Errorf("notes = %v, want expected string", got.Notes)
- }
-}
-
-// TestHandleRejectRequest_NoNotes_OK verifies POST /reject with empty body
-// returns 200 (notes are optional).
-func TestHandleRejectRequest_NoNotes_OK(t *testing.T) {
- h, _ := testHandlers(t)
- resetRequests(t, h)
-
- alice := seedUser(t, h.pool, "alice", "pw", false)
- admin := seedUser(t, h.pool, "admin", "pw", true)
-
- rv := seedPendingArtistRequest(t, h, alice, "artist-mbid-reject-nonotes", "Reject No Notes")
-
- w := doAdminRequestReq(t, h, http.MethodPost, "/api/admin/requests/"+uuidToString(rv.ID)+"/reject", nil, admin)
- if w.Code != http.StatusOK {
- t.Fatalf("status = %d, want 200; body = %s", w.Code, w.Body.String())
- }
-
- var got requestView
- if err := json.Unmarshal(w.Body.Bytes(), &got); err != nil {
- t.Fatalf("decode: %v; body = %s", err, w.Body.String())
- }
- if got.Status != "rejected" {
- t.Errorf("status = %q, want rejected", got.Status)
- }
-}
-
-// TestAdminRequestsEndpoints_NonAdmin403 is a table-driven test verifying that
-// all 3 admin request endpoints reject non-admin users with 403.
-func TestAdminRequestsEndpoints_NonAdmin403(t *testing.T) {
- h, _ := testHandlers(t)
- resetRequests(t, h)
-
- nonAdmin := seedUser(t, h.pool, "regular", "pw", false)
-
- // Use a placeholder UUID for the ID endpoints.
- fakeID := "00000000-0000-0000-0000-000000000001"
-
- cases := []struct {
- method string
- path string
- body []byte
- }{
- {http.MethodGet, "/api/admin/requests", nil},
- {http.MethodPost, "/api/admin/requests/" + fakeID + "/approve", nil},
- {http.MethodPost, "/api/admin/requests/" + fakeID + "/reject", nil},
- }
-
- for _, tc := range cases {
- tc := tc
- t.Run(tc.method+" "+tc.path, func(t *testing.T) {
- w := doAdminRequestReq(t, h, tc.method, tc.path, tc.body, nonAdmin)
- if w.Code != http.StatusForbidden {
- t.Fatalf("status = %d, want 403; body = %s", w.Code, w.Body.String())
- }
- var resp map[string]string
- if err := json.Unmarshal(w.Body.Bytes(), &resp); err != nil {
- t.Fatalf("decode: %v; body = %s", err, w.Body.String())
- }
- if resp["error"] != "not_authorized" {
- t.Errorf("error = %q, want not_authorized", resp["error"])
- }
- })
- }
-}
-
-// TestHandleApproveRequest_AlbumBecomesAWant: approving an album request puts
-// its release group on the wanted list, as the requester's.
-func TestHandleApproveRequest_AlbumBecomesAWant(t *testing.T) {
- h, _ := testHandlers(t)
- resetRequests(t, h)
- alice := seedUser(t, h.pool, "alice", "pw", false)
- admin := seedUser(t, h.pool, "admin", "pw", true)
-
- w := doCreateRequest(h, alice, `{"kind":"album","artist_mbid":"artist-x","artist_name":"Gorillaz","album_mbid":"group-x","album_title":"Demon Days"}`)
- if w.Code != http.StatusCreated {
- t.Fatalf("create: %d %s", w.Code, w.Body.String())
- }
- var rv requestView
- _ = json.Unmarshal(w.Body.Bytes(), &rv)
-
- resp := doAdminRequestReq(t, h, http.MethodPost, "/api/admin/requests/"+uuidToString(rv.ID)+"/approve", nil, admin)
- if resp.Code != http.StatusOK {
- t.Fatalf("approve: %d %s", resp.Code, resp.Body.String())
- }
- want, err := dbq.New(h.pool).GetWantByReleaseGroup(context.Background(), "group-x")
- if err != nil {
- t.Fatalf("no want recorded: %v", err)
- }
- if want.Title != "Demon Days" || want.CreatedBy != alice.ID || want.Reason != "request" {
- t.Errorf("want = %+v", want)
- }
-}
-
-// TestHandleApproveRequest_ArtistOnTheChosenProfile: an artist request is
-// followed on the profile the admin picked; an unknown profile is a 400.
-func TestHandleApproveRequest_ArtistOnTheChosenProfile(t *testing.T) {
- h, _ := testHandlers(t)
- resetRequests(t, h)
- alice := seedUser(t, h.pool, "alice", "pw", false)
- admin := seedUser(t, h.pool, "admin", "pw", true)
- ctx := context.Background()
-
- profile, err := dbq.New(h.pool).CreateReleaseTypeProfile(ctx, dbq.CreateReleaseTypeProfileParams{
- Name: "Albums only", PrimaryTypes: []string{"Album"}, SecondaryTypes: []string{},
- })
- if err != nil {
- t.Fatal(err)
- }
- rv := seedPendingArtistRequest(t, h, alice, "artist-profiled", "Profiled")
- path := "/api/admin/requests/" + uuidToString(rv.ID) + "/approve"
-
- if resp := doAdminRequestReq(t, h, http.MethodPost, path, []byte(`{"profile_id":"00000000-0000-4000-8000-000000000000"}`), admin); resp.Code != http.StatusBadRequest {
- t.Fatalf("unknown profile: %d %s", resp.Code, resp.Body.String())
- }
- body := []byte(`{"profile_id":"` + uuidToString(profile.ID) + `"}`)
- if resp := doAdminRequestReq(t, h, http.MethodPost, path, body, admin); resp.Code != http.StatusOK {
- t.Fatalf("approve: %d %s", resp.Code, resp.Body.String())
- }
- f, err := dbq.New(h.pool).GetFollow(ctx, "artist-profiled")
- if err != nil || f.ProfileID != profile.ID || !f.WantExisting {
- t.Errorf("follow = %+v, %v: on Albums only, wanting what exists", f, err)
- }
-}
diff --git a/internal/api/admin_search.go b/internal/api/admin_search.go
deleted file mode 100644
index aaebdb50..00000000
--- a/internal/api/admin_search.go
+++ /dev/null
@@ -1,340 +0,0 @@
-package api
-
-import (
- "context"
- "encoding/json"
- "errors"
- "net/http"
- "slices"
- "time"
-
- "github.com/go-chi/chi/v5"
- "github.com/jackc/pgx/v5"
-
- "git.fabledsword.com/bvandeusen/minstrel/internal/acquire"
- "git.fabledsword.com/bvandeusen/minstrel/internal/apierror"
- "git.fabledsword.com/bvandeusen/minstrel/internal/db/dbq"
- "git.fabledsword.com/bvandeusen/minstrel/internal/release"
- "git.fabledsword.com/bvandeusen/minstrel/internal/wanted"
-)
-
-// Searching for wants and grabbing releases: the schedule's
-// settings, each want's candidates, and a person's Search, Grab and "never
-// this one".
-
-// searchTimeout bounds a search a person waits on: a Prowlarr fanning out to
-// many trackers can take most of a minute.
-const searchTimeout = 2 * time.Minute
-
-type searchSettingsView struct {
- AutoGrab bool `json:"auto_grab"`
- SearchIntervalHours int32 `json:"search_interval_hours"`
- AllowedQualities []string `json:"allowed_qualities"`
- Tiers []release.TierInfo `json:"tiers"`
- LastPassAt *string `json:"last_pass_at"`
- LastPass json.RawMessage `json:"last_pass"`
- // ImportRoot is the library folder new music is filed into; ""
- // is the first of ScanPaths, the library's folders.
- ImportRoot string `json:"import_root"`
- ScanPaths []string `json:"scan_paths"`
- // The download lifecycle: seed an imported download to this
- // ratio or for this many days; give up on one nobody shares after this
- // many hours. 0 is: remove once imported, no time limit, never.
- SeedRatio float32 `json:"seed_ratio"`
- SeedDays int32 `json:"seed_days"`
- StallHours int32 `json:"stall_hours"`
- // UpgradeUntil is the tier a held album is upgraded to; "" is
- // no quality upgrades.
- UpgradeUntil string `json:"upgrade_until"`
-}
-
-func searchSettings(s dbq.AcquisitionSetting, roots []string) searchSettingsView {
- v := searchSettingsView{
- AutoGrab: s.AutoGrab, SearchIntervalHours: s.SearchIntervalHours,
- AllowedQualities: s.AllowedQualities, Tiers: release.Tiers, LastPass: s.LastPass,
- ImportRoot: s.ImportRoot, ScanPaths: roots,
- SeedRatio: s.SeedRatio, SeedDays: s.SeedDays, StallHours: s.StallHours,
- UpgradeUntil: s.UpgradeUntil,
- }
- if v.ScanPaths == nil {
- v.ScanPaths = []string{}
- }
- if v.AllowedQualities == nil {
- v.AllowedQualities = []string{}
- }
- if len(v.LastPass) == 0 {
- v.LastPass = json.RawMessage("{}")
- }
- if s.LastPassAt.Valid {
- at := formatTimestamp(s.LastPassAt)
- v.LastPassAt = &at
- }
- return v
-}
-
-// handleGetSearchSettings implements GET /api/admin/acquisition/settings.
-func (h *handlers) handleGetSearchSettings(w http.ResponseWriter, r *http.Request) {
- s, err := dbq.New(h.pool).GetAcquisitionSettings(r.Context())
- if err != nil {
- writeErrWithLog(w, h.logger, "admin search settings: get", apierror.Internal(err))
- return
- }
- writeJSON(w, http.StatusOK, searchSettings(s, h.importer.Roots()))
-}
-
-type putSearchSettingsBody struct {
- AutoGrab bool `json:"auto_grab"`
- SearchIntervalHours int32 `json:"search_interval_hours"`
- AllowedQualities []string `json:"allowed_qualities"`
- ImportRoot string `json:"import_root"`
- SeedRatio float32 `json:"seed_ratio"`
- SeedDays int32 `json:"seed_days"`
- StallHours int32 `json:"stall_hours"`
- UpgradeUntil string `json:"upgrade_until"`
-}
-
-// handlePutSearchSettings implements PUT /api/admin/acquisition/settings.
-func (h *handlers) handlePutSearchSettings(w http.ResponseWriter, r *http.Request) {
- var body putSearchSettingsBody
- if !decodeBody(w, r, &body) {
- return
- }
- if body.SearchIntervalHours < 0 || body.SearchIntervalHours > 720 {
- writeErr(w, apierror.BadRequest("bad_interval", "search_interval_hours must be between 0 and 720"))
- return
- }
- if len(body.AllowedQualities) == 0 {
- writeErr(w, apierror.BadRequest("bad_quality", "allow at least one quality"))
- return
- }
- for _, t := range body.AllowedQualities {
- if !release.KnownTier(t) {
- writeErr(w, apierror.BadRequest("bad_quality", "unknown quality "+t))
- return
- }
- }
- if body.UpgradeUntil != "" && !release.KnownTier(body.UpgradeUntil) {
- writeErr(w, apierror.BadRequest("bad_upgrade_until", "unknown quality "+body.UpgradeUntil))
- return
- }
- if body.ImportRoot != "" && !slices.Contains(h.importer.Roots(), body.ImportRoot) {
- writeErr(w, apierror.BadRequest("bad_import_root", "file new music into one of the library's folders"))
- return
- }
- if body.SeedRatio < 0 || body.SeedRatio > 100 || body.SeedDays < 0 || body.SeedDays > 365 ||
- body.StallHours < 0 || body.StallHours > 720 {
- writeErr(w, apierror.BadRequest("bad_lifecycle",
- "seed ratio must be 0 to 100, seed days 0 to 365 and stall hours 0 to 720"))
- return
- }
- s, err := dbq.New(h.pool).UpdateAcquisitionSettings(r.Context(), dbq.UpdateAcquisitionSettingsParams{
- AutoGrab: body.AutoGrab, SearchIntervalHours: body.SearchIntervalHours, AllowedQualities: body.AllowedQualities,
- ImportRoot: body.ImportRoot, SeedRatio: body.SeedRatio, SeedDays: body.SeedDays, StallHours: body.StallHours,
- UpgradeUntil: body.UpgradeUntil,
- })
- if err != nil {
- writeErrWithLog(w, h.logger, "admin search settings: put", apierror.Internal(err))
- return
- }
- writeJSON(w, http.StatusOK, searchSettings(s, h.importer.Roots()))
-}
-
-// candidateView is one release a want's last search found.
-type candidateView struct {
- Key string `json:"key"`
- Name string `json:"name"`
- InfoHash string `json:"info_hash"`
- Indexer string `json:"indexer"`
- SizeBytes int64 `json:"size_bytes"`
- Seeders int32 `json:"seeders"`
- FilesCount int32 `json:"files_count"`
- Format string `json:"format"`
- Quality string `json:"quality"`
- Year int32 `json:"year"`
- Verdict string `json:"verdict"`
- Reason string `json:"reason"`
- Reasons json.RawMessage `json:"reasons"`
- FoldedFrom []string `json:"folded_from"`
- SearchedAt string `json:"searched_at"`
- Downloading bool `json:"downloading"`
-}
-
-// handleListCandidates implements GET /api/admin/wanted/{id}/candidates.
-func (h *handlers) handleListCandidates(w http.ResponseWriter, r *http.Request) {
- id, ok := requireURLUUID(w, r, "id")
- if !ok {
- return
- }
- q := dbq.New(h.pool)
- rows, err := q.ListWantCandidates(r.Context(), id)
- if err != nil {
- writeErrWithLog(w, h.logger, "admin wanted: candidates", apierror.Internal(err))
- return
- }
- live, err := q.GetLiveGrab(r.Context(), id)
- if err != nil && !errors.Is(err, pgx.ErrNoRows) {
- writeErrWithLog(w, h.logger, "admin wanted: live grab", apierror.Internal(err))
- return
- }
- out := make([]candidateView, 0, len(rows))
- for _, c := range rows {
- v := candidateView{
- Key: c.Key, Name: c.Name, InfoHash: c.InfoHash, Indexer: c.Indexer, SizeBytes: c.SizeBytes,
- Seeders: c.Seeders, FilesCount: c.FilesCount, Format: c.Format, Quality: c.Quality, Year: c.Year,
- Verdict: c.Verdict, Reason: c.Reason, Reasons: c.Reasons, FoldedFrom: c.FoldedFrom,
- SearchedAt: formatTimestamp(c.SearchedAt),
- Downloading: live.ID.Valid && live.Key == c.Key,
- }
- if v.FoldedFrom == nil {
- v.FoldedFrom = []string{}
- }
- if len(v.Reasons) == 0 {
- v.Reasons = json.RawMessage("[]")
- }
- out = append(out, v)
- }
- writeJSON(w, http.StatusOK, map[string]any{"items": out})
-}
-
-// handleSearchWant implements POST /api/admin/wanted/{id}/search: search
-// now, as the schedule would, taking the best release when auto-grab is on.
-func (h *handlers) handleSearchWant(w http.ResponseWriter, r *http.Request) {
- id, ok := requireURLUUID(w, r, "id")
- if !ok {
- return
- }
- ctx, cancel := context.WithTimeout(r.Context(), searchTimeout)
- defer cancel()
- res, err := h.searcher.SearchNow(ctx, id)
- if h.writeSearchErr(w, err) {
- return
- }
- writeJSON(w, http.StatusOK, res)
-}
-
-type grabBody struct {
- Key string `json:"key"`
- Force bool `json:"force"`
-}
-
-// grabView is what came of a person's grab.
-type grabView struct {
- Added bool `json:"added"`
- Refused bool `json:"refused"`
- Gone bool `json:"gone"`
- Unconfigured bool `json:"unconfigured"`
- Missing string `json:"missing,omitempty"`
- Kept bool `json:"kept"`
- Replaced bool `json:"replaced"`
- Verdict string `json:"verdict,omitempty"`
- Reason string `json:"reason,omitempty"`
- Error string `json:"error,omitempty"`
-}
-
-// handleGrabCandidate implements POST /api/admin/wanted/{id}/grab: take the
-// release named by key. The search runs again first, so the indexer's link
-// never leaves the server; force takes what the verdict would not.
-func (h *handlers) handleGrabCandidate(w http.ResponseWriter, r *http.Request) {
- admin, ok := requireUser(w, r)
- if !ok {
- return
- }
- id, ok := requireURLUUID(w, r, "id")
- if !ok {
- return
- }
- var body grabBody
- if !decodeBody(w, r, &body) {
- return
- }
- ctx, cancel := context.WithTimeout(r.Context(), searchTimeout)
- defer cancel()
- took, err := h.searcher.Take(ctx, id, body.Key, body.Force, admin.ID)
- if h.writeSearchErr(w, err) {
- return
- }
- v := grabView{
- Added: took.Added, Refused: took.Refused, Gone: took.Gone, Unconfigured: took.Unconfigured,
- Missing: took.Missing, Error: took.Error,
- }
- if took.Chosen != nil {
- v.Verdict, v.Reason = string(took.Chosen.Verdict), took.Chosen.Reason
- }
- if took.Contest != nil {
- v.Kept, v.Replaced = took.Contest.Kept, took.Contest.Replaced
- if took.Contest.Kept {
- v.Reason = took.Contest.Reason
- }
- }
- writeJSON(w, http.StatusOK, v)
-}
-
-type blockBody struct {
- Key string `json:"key"`
-}
-
-// handleBlockCandidate implements POST /api/admin/wanted/{id}/block: never
-// take this release again, for any want.
-func (h *handlers) handleBlockCandidate(w http.ResponseWriter, r *http.Request) {
- admin, ok := requireUser(w, r)
- if !ok {
- return
- }
- id, ok := requireURLUUID(w, r, "id")
- if !ok {
- return
- }
- var body blockBody
- if !decodeBody(w, r, &body) {
- return
- }
- c, err := dbq.New(h.pool).GetWantCandidate(r.Context(), dbq.GetWantCandidateParams{WantID: id, Key: body.Key})
- if errors.Is(err, pgx.ErrNoRows) {
- writeErr(w, apierror.NotFound("candidate"))
- return
- }
- if err != nil {
- writeErrWithLog(w, h.logger, "admin wanted: block", apierror.Internal(err))
- return
- }
- if c.InfoHash == "" {
- writeErr(w, apierror.BadRequest("no_infohash", "this release has no infohash to set aside by"))
- return
- }
- if err := acquire.NewMemory(h.pool).Block(r.Context(), dbq.BlockReleaseParams{
- InfoHash: c.InfoHash, Name: c.Name, Reason: acquire.BlockOperator, WantID: id, BlockedBy: admin.ID,
- }); err != nil {
- writeErrWithLog(w, h.logger, "admin wanted: block", apierror.Internal(err))
- return
- }
- w.WriteHeader(http.StatusNoContent)
-}
-
-// handleUnblockRelease implements DELETE /api/admin/release-blocks/{hash}.
-func (h *handlers) handleUnblockRelease(w http.ResponseWriter, r *http.Request) {
- removed, err := acquire.NewMemory(h.pool).Unblock(r.Context(), chi.URLParam(r, "hash"))
- if err != nil {
- writeErrWithLog(w, h.logger, "admin wanted: unblock", apierror.Internal(err))
- return
- }
- if !removed {
- writeErr(w, apierror.NotFound("block"))
- return
- }
- w.WriteHeader(http.StatusNoContent)
-}
-
-// writeSearchErr answers a Searcher error, and says whether there was one.
-func (h *handlers) writeSearchErr(w http.ResponseWriter, err error) bool {
- switch {
- case err == nil:
- return false
- case errors.Is(err, pgx.ErrNoRows):
- writeErr(w, apierror.NotFound("want"))
- case errors.Is(err, wanted.ErrWantHeld):
- writeErr(w, apierror.Conflict("want_held", "the library already holds this album, and it isn't wanted for a better copy"))
- default:
- writeErrWithLog(w, h.logger, "admin wanted: search", apierror.Internal(err))
- }
- return true
-}
diff --git a/internal/api/admin_status.go b/internal/api/admin_status.go
index 17c6f8fa..e7866907 100644
--- a/internal/api/admin_status.go
+++ b/internal/api/admin_status.go
@@ -11,7 +11,7 @@ import (
)
// The admin Overview's status sections: after the verdict, who is
-// listening, what is downloading, what the library holds and how much room
+// listening, what the library holds and how much room
// it has, and how the server itself is doing. One read for the page, polled
// while it is open.
@@ -36,17 +36,6 @@ type listeningStatusView struct {
LastPlayAt *string `json:"last_play_at"`
}
-type downloadsStatusView struct {
- Running int64 `json:"running"`
- NeedsYou int64 `json:"needs_you"`
- NeedsYouSince *string `json:"needs_you_since"`
- ImportedDay int64 `json:"imported_day"`
- GivenUpDay int64 `json:"given_up_day"`
- // Live is the downloads not yet ended, the ones needing a person first.
- Live []downloadView `json:"live"`
- ClientsDown []clientDownView `json:"clients_down"`
-}
-
// diskView is the filesystem one library folder, or the data folder, sits
// on.
type diskView struct {
@@ -91,15 +80,10 @@ type serverStatusView struct {
type adminStatusView struct {
Listening listeningStatusView `json:"listening"`
- Downloads downloadsStatusView `json:"downloads"`
Library libraryStatusView `json:"library"`
Server serverStatusView `json:"server"`
}
-// overviewLiveDownloads is how many running downloads the Overview lists;
-// Activity has the rest.
-const overviewLiveDownloads = 5
-
// overviewErrors is how many recent errors the Overview lists.
const overviewErrors = 10
@@ -136,12 +120,6 @@ func (h *handlers) handleGetAdminStatus(w http.ResponseWriter, r *http.Request)
out.Listening.LastPlayAt = &at
}
- if err := h.downloadsStatus(r, q, &out.Downloads); err != nil {
- h.logger.Error("admin status: downloads", "err", err)
- writeAdminJSONErr(w, http.StatusInternalServerError, "server_error")
- return
- }
-
lib, err := q.LibraryStatus(ctx)
if err != nil {
h.logger.Error("admin status: library", "err", err)
@@ -150,8 +128,8 @@ func (h *handlers) handleGetAdminStatus(w http.ResponseWriter, r *http.Request)
}
out.Library = libraryStatusView{Tracks: lib.Tracks, Albums: lib.Albums, Artists: lib.Artists,
Bytes: lib.Bytes, Disks: []diskView{}}
- if h.importer != nil {
- for _, root := range h.importer.Roots() {
+ if h.scanner != nil {
+ for _, root := range h.scanner.Roots() {
out.Library.Disks = append(out.Library.Disks, diskOf(root))
}
}
@@ -170,39 +148,6 @@ func (h *handlers) handleGetAdminStatus(w http.ResponseWriter, r *http.Request)
writeJSON(w, http.StatusOK, out)
}
-func (h *handlers) downloadsStatus(r *http.Request, q *dbq.Queries, out *downloadsStatusView) error {
- st, err := q.DownloadsStatus(r.Context())
- if err != nil {
- return err
- }
- *out = downloadsStatusView{Running: st.Running, NeedsYou: st.NeedsYou,
- ImportedDay: st.ImportedDay, GivenUpDay: st.GivenUpDay,
- Live: []downloadView{}, ClientsDown: []clientDownView{}}
- if st.NeedsYou > 0 && st.NeedsYouSince.Valid {
- since := formatTimestamp(st.NeedsYouSince)
- out.NeedsYouSince = &since
- }
- // ListGrabs puts the ones needing a person first, then the running ones.
- rows, err := q.ListGrabs(r.Context(), dbq.ListGrabsParams{MaxRows: overviewLiveDownloads})
- if err != nil {
- return err
- }
- for _, g := range rows {
- if !g.EndedAt.Valid {
- out.Live = append(out.Live, downloadFrom(g))
- }
- }
- down, err := q.ListClientsDown(r.Context())
- if err != nil {
- return err
- }
- for _, c := range down {
- out.ClientsDown = append(out.ClientsDown, clientDownView{Client: c.Client, Detail: c.Detail,
- Since: formatTimestamp(c.FirstSeen)})
- }
- return nil
-}
-
// diskOf reads the size and free space of the filesystem path is on.
func diskOf(path string) diskView {
var st syscall.Statfs_t
diff --git a/internal/api/admin_status_test.go b/internal/api/admin_status_test.go
index 4e2bcc07..cc76037e 100644
--- a/internal/api/admin_status_test.go
+++ b/internal/api/admin_status_test.go
@@ -76,7 +76,7 @@ func TestAdminStatus_ListeningAndServer(t *testing.T) {
if got.Library.Tracks != 2 || got.Library.Albums != 2 || got.Library.Bytes != 10 {
t.Errorf("library = %+v, want 2 tracks, 2 albums, 10 bytes", got.Library)
}
- if got.Downloads.Live == nil || got.Downloads.ClientsDown == nil || got.Library.Disks == nil {
+ if got.Library.Disks == nil {
t.Error("lists must be [], never null")
}
diff --git a/internal/api/admin_summary.go b/internal/api/admin_summary.go
index 9a870497..98f5c6a5 100644
--- a/internal/api/admin_summary.go
+++ b/internal/api/admin_summary.go
@@ -19,7 +19,6 @@ type adminQueueView struct {
// every queue an admin acts on, in one response, so each admin page pays one
// cheap read for its rail.
type adminSummaryView struct {
- Requests adminQueueView `json:"requests"`
Reports adminQueueView `json:"reports"`
Duplicates adminQueueView `json:"duplicates"`
Suspect adminQueueView `json:"suspect"`
@@ -37,7 +36,6 @@ func queueView(count int64, since pgtype.Timestamptz) adminQueueView {
func adminSummaryFrom(row dbq.AdminSummaryRow) adminSummaryView {
return adminSummaryView{
- Requests: queueView(row.Requests, row.RequestsSince),
Reports: queueView(row.Reports, row.ReportsSince),
Duplicates: queueView(row.Duplicates, row.DuplicatesSince),
Suspect: queueView(row.Suspect, row.SuspectSince),
diff --git a/internal/api/admin_summary_test.go b/internal/api/admin_summary_test.go
index f6970512..dffa9b69 100644
--- a/internal/api/admin_summary_test.go
+++ b/internal/api/admin_summary_test.go
@@ -35,6 +35,7 @@ func getAdminSummary(t *testing.T, h *handlers, user dbq.User) (int, adminSummar
// oldest item has waited; an empty queue has no since.
func TestAdminSummary_CountsTheQueuesAndSinceWhen(t *testing.T) {
h, _ := testHandlers(t)
+ truncateLibrary(t, h.pool)
admin := seedUser(t, h.pool, "admin", "pw", true)
listener := seedUser(t, h.pool, "listener", "pw", false)
ctx := context.Background()
@@ -43,19 +44,17 @@ func TestAdminSummary_CountsTheQueuesAndSinceWhen(t *testing.T) {
if code != http.StatusOK {
t.Fatalf("status = %d", code)
}
- if empty.Requests.Count != 0 || empty.Requests.Since != nil || empty.Missing.Count != 0 {
+ if empty.Missing.Count != 0 || empty.Missing.Since != nil {
t.Errorf("empty summary = %+v", empty)
}
- if _, err := dbq.New(h.pool).CreateRequest(ctx, dbq.CreateRequestParams{
- UserID: listener.ID, Kind: dbq.RequestKindArtist, ArtistMbid: "a0000000-0000-4000-8000-000000000001",
- ArtistName: "Annadale",
- }); err != nil {
+ track := seedQuarantineTrack(t, h, "sum-missing")
+ if _, err := h.pool.Exec(ctx, `UPDATE tracks SET missing_since = now() WHERE id = $1`, track.ID); err != nil {
t.Fatal(err)
}
_, got := getAdminSummary(t, h, admin)
- if got.Requests.Count != 1 || got.Requests.Since == nil {
- t.Errorf("requests = %+v, want 1 with a since", got.Requests)
+ if got.Missing.Count != 1 || got.Missing.Since == nil {
+ t.Errorf("missing = %+v, want 1 with a since", got.Missing)
}
if code, _ := getAdminSummary(t, h, listener); code != http.StatusForbidden {
diff --git a/internal/api/admin_suspect_sources.go b/internal/api/admin_suspect_sources.go
index 8e5a0e90..7e51109b 100644
--- a/internal/api/admin_suspect_sources.go
+++ b/internal/api/admin_suspect_sources.go
@@ -31,10 +31,6 @@ type suspectTrackView struct {
// first pass. VerdictAt is when it was last set.
Verdict *string `json:"verdict"`
VerdictAt *string `json:"verdict_at"`
- // Replacing: the album is on the wanted list for a clean copy;
- // SearchedAt is when one was last searched for, null before the first.
- Replacing bool `json:"replacing"`
- SearchedAt *string `json:"searched_at"`
}
// suspectGroupView is a folder's worth of flagged tracks. As on the
@@ -116,11 +112,6 @@ func groupSuspectByDirectory(rows []dbq.ListSuspectSourceTracksRow) []suspectGro
at := row.SourceVerdictAt.Time.UTC().Format(time.RFC3339)
t.VerdictAt = &at
}
- t.Replacing = row.WantUpgrade == "rip"
- if t.Replacing && row.SearchedAt.Valid {
- at := row.SearchedAt.Time.UTC().Format(time.RFC3339)
- t.SearchedAt = &at
- }
if n := len(groups); n > 0 && groups[n-1].Directory == row.Directory {
groups[n-1].Tracks = append(groups[n-1].Tracks, t)
continue
diff --git a/internal/api/admin_users.go b/internal/api/admin_users.go
index 9c729e79..6115e9dd 100644
--- a/internal/api/admin_users.go
+++ b/internal/api/admin_users.go
@@ -17,13 +17,12 @@ import (
)
type adminUserView struct {
- ID string `json:"id"`
- Username string `json:"username"`
- DisplayName *string `json:"display_name"`
- IsAdmin bool `json:"is_admin"`
- AutoApproveRequests bool `json:"auto_approve_requests"`
- DebugModeEnabled bool `json:"debug_mode_enabled"`
- CreatedAt string `json:"created_at"`
+ ID string `json:"id"`
+ Username string `json:"username"`
+ DisplayName *string `json:"display_name"`
+ IsAdmin bool `json:"is_admin"`
+ DebugModeEnabled bool `json:"debug_mode_enabled"`
+ CreatedAt string `json:"created_at"`
}
type listUsersResp struct {
@@ -40,13 +39,12 @@ func (h *handlers) handleAdminListUsers(w http.ResponseWriter, r *http.Request)
out := make([]adminUserView, 0, len(rows))
for _, row := range rows {
v := adminUserView{
- ID: uuidToString(row.ID),
- Username: row.Username,
- DisplayName: row.DisplayName,
- IsAdmin: row.IsAdmin,
- AutoApproveRequests: row.AutoApproveRequests,
- DebugModeEnabled: row.DebugModeEnabled,
- CreatedAt: row.CreatedAt.Time.UTC().Format(time.RFC3339),
+ ID: uuidToString(row.ID),
+ Username: row.Username,
+ DisplayName: row.DisplayName,
+ IsAdmin: row.IsAdmin,
+ DebugModeEnabled: row.DebugModeEnabled,
+ CreatedAt: row.CreatedAt.Time.UTC().Format(time.RFC3339),
}
out = append(out, v)
}
@@ -117,13 +115,12 @@ func (h *handlers) handleUpdateUserAdmin(w http.ResponseWriter, r *http.Request)
audit.WriteOrLog(r.Context(), h.pool, h.logger, caller.ID, targetID, action, nil)
writeJSON(w, http.StatusOK, adminUserView{
- ID: uuidToString(updated.ID),
- Username: updated.Username,
- DisplayName: updated.DisplayName,
- IsAdmin: updated.IsAdmin,
- AutoApproveRequests: updated.AutoApproveRequests,
- DebugModeEnabled: updated.DebugModeEnabled,
- CreatedAt: updated.CreatedAt.Time.UTC().Format(time.RFC3339),
+ ID: uuidToString(updated.ID),
+ Username: updated.Username,
+ DisplayName: updated.DisplayName,
+ IsAdmin: updated.IsAdmin,
+ DebugModeEnabled: updated.DebugModeEnabled,
+ CreatedAt: updated.CreatedAt.Time.UTC().Format(time.RFC3339),
})
}
@@ -187,13 +184,12 @@ func (h *handlers) handleAdminCreateUser(w http.ResponseWriter, r *http.Request)
map[string]any{"is_admin": req.IsAdmin})
writeJSON(w, http.StatusOK, adminUserView{
- ID: uuidToString(user.ID),
- Username: user.Username,
- DisplayName: user.DisplayName,
- IsAdmin: user.IsAdmin,
- AutoApproveRequests: user.AutoApproveRequests,
- DebugModeEnabled: user.DebugModeEnabled,
- CreatedAt: user.CreatedAt.Time.UTC().Format(time.RFC3339),
+ ID: uuidToString(user.ID),
+ Username: user.Username,
+ DisplayName: user.DisplayName,
+ IsAdmin: user.IsAdmin,
+ DebugModeEnabled: user.DebugModeEnabled,
+ CreatedAt: user.CreatedAt.Time.UTC().Format(time.RFC3339),
})
}
@@ -302,51 +298,3 @@ func (h *handlers) handleAdminResetPassword(w http.ResponseWriter, r *http.Reque
w.WriteHeader(http.StatusNoContent)
}
-
-// --- Admin auto-approve toggle ---------------------------------------------
-
-type adminAutoApproveReq struct {
- AutoApprove bool `json:"auto_approve"`
-}
-
-func (h *handlers) handleAdminAutoApproveToggle(w http.ResponseWriter, r *http.Request) {
- caller, ok := requireUser(w, r)
- if !ok {
- return
- }
- targetID, ok := requireURLUUID(w, r, "id")
- if !ok {
- return
- }
- var req adminAutoApproveReq
- if !decodeBody(w, r, &req) {
- return
- }
-
- q := dbq.New(h.pool)
- updated, err := q.UpdateUserAutoApprove(r.Context(), dbq.UpdateUserAutoApproveParams{
- ID: targetID,
- AutoApproveRequests: req.AutoApprove,
- })
- if err != nil {
- if errors.Is(err, pgx.ErrNoRows) {
- writeErr(w, &apierror.Error{Status: http.StatusNotFound, Code: "not_found", Message: ""})
- return
- }
- writeErrWithLog(w, h.logger, "admin auto-approve: update failed", apierror.Internal(err))
- return
- }
-
- audit.WriteOrLog(r.Context(), h.pool, h.logger, caller.ID, targetID, audit.ActionAutoApproveToggle,
- map[string]any{"auto_approve": req.AutoApprove})
-
- writeJSON(w, http.StatusOK, adminUserView{
- ID: uuidToString(updated.ID),
- Username: updated.Username,
- DisplayName: updated.DisplayName,
- IsAdmin: updated.IsAdmin,
- AutoApproveRequests: updated.AutoApproveRequests,
- DebugModeEnabled: updated.DebugModeEnabled,
- CreatedAt: updated.CreatedAt.Time.UTC().Format(time.RFC3339),
- })
-}
diff --git a/internal/api/admin_users_test.go b/internal/api/admin_users_test.go
index 6895fb72..5536ce47 100644
--- a/internal/api/admin_users_test.go
+++ b/internal/api/admin_users_test.go
@@ -25,7 +25,6 @@ func newAdminUsersRouter(h *handlers) chi.Router {
admin.Post("/users", h.handleAdminCreateUser)
admin.Delete("/users/{id}", h.handleAdminDeleteUser)
admin.Post("/users/{id}/reset-password", h.handleAdminResetPassword)
- admin.Put("/users/{id}/auto-approve", h.handleAdminAutoApproveToggle)
})
return r
}
@@ -271,40 +270,3 @@ func TestAdminResetPassword_TooShort_400(t *testing.T) {
t.Errorf("status = %d, want 400", rec.Code)
}
}
-
-func TestAdminAutoApproveToggle(t *testing.T) {
- if os.Getenv("MINSTREL_TEST_DATABASE_URL") == "" {
- t.Skip("MINSTREL_TEST_DATABASE_URL not set")
- }
- h, pool := testHandlers(t)
- admin := seedUser(t, pool, "aacaller", "pw", true)
- target := seedUser(t, pool, "aatarget", "pw", false)
-
- body := `{"auto_approve":true}`
- req := httptest.NewRequest(http.MethodPut, "/api/admin/users/"+uuidToString(target.ID)+"/auto-approve",
- bytes.NewReader([]byte(body)))
- req = withUser(req, admin)
- rec := httptest.NewRecorder()
- newAdminUsersRouter(h).ServeHTTP(rec, req)
-
- if rec.Code != http.StatusOK {
- t.Fatalf("status = %d, want 200; body=%s", rec.Code, rec.Body.String())
- }
- var resp adminUserView
- if err := json.Unmarshal(rec.Body.Bytes(), &resp); err != nil {
- t.Fatalf("decode: %v", err)
- }
- if !resp.AutoApproveRequests {
- t.Errorf("auto_approve_requests = false on response; expected true")
- }
-
- // Verify DB.
- var aa bool
- if err := pool.QueryRow(context.Background(),
- "SELECT auto_approve_requests FROM users WHERE id = $1", target.ID).Scan(&aa); err != nil {
- t.Fatalf("verify: %v", err)
- }
- if !aa {
- t.Errorf("DB row not updated")
- }
-}
diff --git a/internal/api/admin_wanted.go b/internal/api/admin_wanted.go
deleted file mode 100644
index 0cb28b80..00000000
--- a/internal/api/admin_wanted.go
+++ /dev/null
@@ -1,821 +0,0 @@
-package api
-
-import (
- "context"
- "encoding/json"
- "errors"
- "net/http"
- "strconv"
- "strings"
-
- "github.com/go-chi/chi/v5"
- "github.com/jackc/pgx/v5"
- "github.com/jackc/pgx/v5/pgtype"
-
- "git.fabledsword.com/bvandeusen/minstrel/internal/apierror"
- "git.fabledsword.com/bvandeusen/minstrel/internal/catalogue"
- "git.fabledsword.com/bvandeusen/minstrel/internal/db/dbq"
- "git.fabledsword.com/bvandeusen/minstrel/internal/musicbrainz"
- "git.fabledsword.com/bvandeusen/minstrel/internal/wanted"
-)
-
-// The wanted list's admin surface: wants, follows and
-// release-type profiles.
-
-// wantView is one row of GET /api/admin/wanted.
-type wantView struct {
- ID string `json:"id"`
- ReleaseGroupMBID string `json:"release_group_mbid"`
- Title string `json:"title"`
- ArtistMBID string `json:"artist_mbid"`
- ArtistName string `json:"artist_name"`
- ArtistID string `json:"artist_id,omitempty"`
- PrimaryType string `json:"primary_type"`
- SecondaryTypes []string `json:"secondary_types"`
- Year string `json:"year"`
- Reason string `json:"reason"`
- RequestedBy string `json:"requested_by,omitempty"`
- AlbumID string `json:"album_id,omitempty"`
- WantedAt string `json:"wanted_at"`
- HeldAt *string `json:"held_at"`
- ImageURL string `json:"image_url"`
- // Upgrade: why a held album is wanted again, quality or rip, and
- // HeldQuality what the library's copy is.
- Upgrade string `json:"upgrade,omitempty"`
- HeldQuality string `json:"held_quality,omitempty"`
- // Search is the schedule's last look; Grab the download running.
- Search *wantSearchView `json:"search"`
- Grab *wantGrabView `json:"grab"`
-}
-
-type wantSearchView struct {
- SearchedAt string `json:"searched_at"`
- Outcome string `json:"outcome"`
- Considered int32 `json:"considered"`
- BestRelease string `json:"best_release"`
- BestVerdict string `json:"best_verdict"`
- Error string `json:"error"`
-}
-
-// wantGrabView is the want's latest download, running or ended, and what
-// its import made of it.
-type wantGrabView struct {
- ID string `json:"id"`
- Name string `json:"name"`
- GrabbedAt string `json:"grabbed_at"`
- Origin string `json:"origin"`
- Quality string `json:"quality"`
- Live bool `json:"live"`
- EndReason string `json:"end_reason"`
- Progress float32 `json:"progress"`
- ImportState string `json:"import_state"`
- ImportReason string `json:"import_reason"`
- EndDetail string `json:"end_detail"`
- ImportedAt *string `json:"imported_at"`
- ImportedFiles int32 `json:"imported_files"`
-}
-
-// wantCounts sizes each tab of the Wanted page. Artists: how many artists
-// the list holds albums of, its main page.
-type wantCounts struct {
- Artists int64 `json:"artists"`
- // Unmatched: library albums waiting to be matched by hand.
- Unmatched int64 `json:"unmatched"`
- Wanted int64 `json:"wanted"`
- Missing int64 `json:"missing"`
- Held int64 `json:"held"`
- // Upgrades: held albums wanted for a better copy.
- Upgrades int64 `json:"upgrades"`
- Following int64 `json:"following"`
- // Activity: downloads running; Attention: of those, the ones waiting on
- // a person; Blocked: releases set aside.
- Activity int64 `json:"activity"`
- Attention int64 `json:"attention"`
- Blocked int64 `json:"blocked"`
-}
-
-type wantsPage struct {
- Items []wantView `json:"items"`
- Total int64 `json:"total"`
- Limit int `json:"limit"`
- Offset int `json:"offset"`
- Counts wantCounts `json:"counts"`
- // Library is the one-time listing of the library's albums.
- Library libraryListingView `json:"library"`
- // Artist heads one artist's page, when the list is narrowed to them.
- Artist *wantArtistView `json:"artist,omitempty"`
-}
-
-// wantArtistView is one artist on the Wanted view's main page: how
-// many of their albums the list holds, of how many.
-type wantArtistView struct {
- ArtistMBID string `json:"artist_mbid"`
- ArtistName string `json:"artist_name"`
- SortName string `json:"sort_name"`
- ArtistID string `json:"artist_id,omitempty"`
- ImageURL string `json:"image_url"`
- Albums int32 `json:"albums"`
- Held int32 `json:"held"`
- Wanted int32 `json:"wanted"`
- Missing int32 `json:"missing"`
- Downloading int32 `json:"downloading"`
- Following bool `json:"following"`
-}
-
-type wantArtistsPage struct {
- Items []wantArtistView `json:"items"`
- Total int64 `json:"total"`
- Limit int `json:"limit"`
- Offset int `json:"offset"`
- Counts wantCounts `json:"counts"`
- Library libraryListingView `json:"library"`
-}
-
-// libraryListingView: whether the library's albums are all on the list
-// yet, and how many could not be matched to MusicBrainz; those are
-// listed in the Collection's Unmatched section.
-type libraryListingView struct {
- ListedAt *string `json:"listed_at"`
- Unlisted int32 `json:"unlisted"`
-}
-
-// unmatchedAlbumView is one album in the Collection's Unmatched section: why
-// no MusicBrainz album matched it, or that a person left it off.
-type unmatchedAlbumView struct {
- AlbumID string `json:"album_id"`
- Title string `json:"title"`
- Artist string `json:"artist"`
- Detail string `json:"detail"`
- Tracks int32 `json:"tracks"`
- LeftOff bool `json:"left_off"`
- DecidedAt string `json:"decided_at"`
-}
-
-// unmatchedShown bounds the Unmatched section: a library a listing could
-// not match more of than this has a bigger problem than one page shows.
-const unmatchedShown = 500
-
-const wantsPageSize = 50
-
-// wantArtistsPageSize fills a few rows of the artist grid a page.
-const wantArtistsPageSize = 96
-
-// wantsHeader is what every Collection page carries: each section's count,
-// and how the one-time library listing went.
-func (h *handlers) wantsHeader(r *http.Request, q *dbq.Queries) (wantCounts, libraryListingView, error) {
- counts, err := q.CountWants(r.Context())
- if err != nil {
- return wantCounts{}, libraryListingView{}, err
- }
- listing, err := q.GetLibraryListing(r.Context())
- if err != nil {
- return wantCounts{}, libraryListingView{}, err
- }
- lib := libraryListingView{Unlisted: listing.LibraryUnlisted}
- if listing.LibraryListedAt.Valid {
- at := formatTimestamp(listing.LibraryListedAt)
- lib.ListedAt = &at
- }
- return wantCounts{Artists: counts.Artists, Unmatched: counts.Unmatched, Wanted: counts.Wanted, Missing: counts.Missing, Held: counts.Held,
- Upgrades: counts.Upgrades, Following: counts.Following, Activity: counts.Activity, Attention: counts.Attention, Blocked: counts.Blocked}, lib, nil
-}
-
-func offsetParam(r *http.Request) int {
- offset, _ := strconv.Atoi(r.URL.Query().Get("offset"))
- return max(offset, 0)
-}
-
-func wantArtistViewOf(row dbq.ListWantArtistsRow) wantArtistView {
- v := wantArtistView{
- ArtistMBID: row.ArtistMbid, ArtistName: row.ArtistName, SortName: row.SortName,
- ImageURL: artistArtURL(row.ArtistMbid, row.ArtistName),
- Albums: row.Albums, Held: row.Held, Wanted: row.Wanted, Missing: row.Missing,
- Downloading: row.Downloading, Following: row.Following,
- }
- if row.LibraryArtistID.Valid {
- v.ArtistID = uuidToString(row.LibraryArtistID)
- }
- return v
-}
-
-// handleListWantArtists implements GET /api/admin/wanted/artists?q=&offset=:
-// the Wanted view's main page, every artist on the list A–Z.
-func (h *handlers) handleListWantArtists(w http.ResponseWriter, r *http.Request) {
- offset := offsetParam(r)
- q := dbq.New(h.pool)
- rows, err := q.ListWantArtists(r.Context(), dbq.ListWantArtistsParams{
- Q: strings.TrimSpace(r.URL.Query().Get("q")), Skip: int32(offset), MaxRows: wantArtistsPageSize})
- if err != nil {
- writeErrWithLog(w, h.logger, "admin wanted: artists", apierror.Internal(err))
- return
- }
- counts, lib, err := h.wantsHeader(r, q)
- if err != nil {
- writeErrWithLog(w, h.logger, "admin wanted: counts", apierror.Internal(err))
- return
- }
- out := wantArtistsPage{Items: make([]wantArtistView, 0, len(rows)), Limit: wantArtistsPageSize, Offset: offset,
- Counts: counts, Library: lib}
- for _, row := range rows {
- out.Total = row.Total
- out.Items = append(out.Items, wantArtistViewOf(row))
- }
- writeJSON(w, http.StatusOK, out)
-}
-
-// handleListWants implements GET
-// /api/admin/wanted?state=wanted|missing|upgrade|held|all&artist=&q=&offset=.
-// wanted is everything open but missing files; upgrade the held albums wanted
-// for a better copy. Each list runs artist by artist, A–Z, like the library; q
-// narrows it to an artist or album, and artist to one artist's page, which it
-// then heads.
-func (h *handlers) handleListWants(w http.ResponseWriter, r *http.Request) {
- state := r.URL.Query().Get("state")
- if state == "" {
- state = "wanted"
- }
- if state != "wanted" && state != "missing" && state != "upgrade" && state != "held" && state != "all" {
- writeErr(w, apierror.BadRequest("bad_state", "state must be wanted, missing, upgrade, held or all"))
- return
- }
- offset := offsetParam(r)
- artist := strings.TrimSpace(r.URL.Query().Get("artist"))
- q := dbq.New(h.pool)
- rows, err := q.ListWants(r.Context(), dbq.ListWantsParams{State: state, ArtistMbid: artist,
- Q: strings.TrimSpace(r.URL.Query().Get("q")), Skip: int32(offset), MaxRows: wantsPageSize})
- if err != nil {
- writeErrWithLog(w, h.logger, "admin wanted: list", apierror.Internal(err))
- return
- }
- counts, lib, err := h.wantsHeader(r, q)
- if err != nil {
- writeErrWithLog(w, h.logger, "admin wanted: counts", apierror.Internal(err))
- return
- }
- out := wantsPage{Library: lib, Counts: counts,
- Items: make([]wantView, 0, len(rows)), Limit: wantsPageSize, Offset: offset}
- if artist != "" {
- heads, err := q.ListWantArtists(r.Context(), dbq.ListWantArtistsParams{ArtistMbid: artist, MaxRows: 1})
- if err != nil {
- writeErrWithLog(w, h.logger, "admin wanted: artist", apierror.Internal(err))
- return
- }
- if len(heads) == 0 {
- writeErr(w, apierror.NotFound("artist"))
- return
- }
- v := wantArtistViewOf(heads[0])
- out.Artist = &v
- }
- for _, row := range rows {
- out.Total = row.Total
- out.Items = append(out.Items, wantViewOf(row))
- }
- writeJSON(w, http.StatusOK, out)
-}
-
-// wantViewOf is one ListWants row as the wanted list and the Missing page
-// send it.
-func wantViewOf(row dbq.ListWantsRow) wantView {
- v := wantView{
- ID: uuidToString(row.ID), ReleaseGroupMBID: row.ReleaseGroupMbid, Title: row.Title,
- ArtistMBID: row.ArtistMbid, ArtistName: row.ArtistName,
- PrimaryType: row.PrimaryType, SecondaryTypes: row.SecondaryTypes,
- Year: first4(row.FirstReleaseDate), Reason: row.Reason,
- WantedAt: formatTimestamp(row.WantedAt), Upgrade: row.Upgrade, HeldQuality: row.HeldQuality,
- ImageURL: groupArtURL(musicbrainz.ReleaseGroup{MBID: row.ReleaseGroupMbid, Title: row.Title, ArtistCredit: row.ArtistName}),
- }
- if row.LibraryArtistID.Valid {
- v.ArtistID = uuidToString(row.LibraryArtistID)
- }
- if row.AlbumID.Valid {
- v.AlbumID = uuidToString(row.AlbumID)
- }
- if row.RequestedBy != nil {
- v.RequestedBy = *row.RequestedBy
- }
- if row.FulfilledAt.Valid {
- held := formatTimestamp(row.FulfilledAt)
- v.HeldAt = &held
- }
- if v.SecondaryTypes == nil {
- v.SecondaryTypes = []string{}
- }
- if row.SearchedAt.Valid {
- v.Search = &wantSearchView{
- SearchedAt: formatTimestamp(row.SearchedAt), Outcome: row.SearchOutcome,
- Considered: row.SearchConsidered, BestRelease: row.BestRelease,
- BestVerdict: row.BestVerdict, Error: row.SearchError,
- }
- }
- if row.GrabName != nil {
- v.Grab = &wantGrabView{ID: uuidToString(row.GrabID), Name: *row.GrabName,
- GrabbedAt: formatTimestamp(row.GrabbedAt), Live: !row.GrabEndedAt.Valid,
- Origin: deref(row.GrabOrigin), Quality: deref(row.GrabQuality), EndReason: deref(row.GrabEndReason),
- ImportState: deref(row.GrabImportState), ImportReason: deref(row.GrabImportReason),
- EndDetail: deref(row.GrabEndDetail)}
- if row.GrabProgress != nil {
- v.Grab.Progress = *row.GrabProgress
- }
- if row.GrabImportedFiles != nil {
- v.Grab.ImportedFiles = *row.GrabImportedFiles
- }
- if row.GrabImportedAt.Valid {
- at := formatTimestamp(row.GrabImportedAt)
- v.Grab.ImportedAt = &at
- }
- }
- return v
-}
-
-// wantTracksView is an album's track list for the Wanted view: the
-// library's own tracks for an album it holds or held, else MusicBrainz's
-// standard edition of what is wanted.
-type wantTracksView struct {
- // library, musicbrainz, or none when MusicBrainz lists no edition.
- Source string `json:"source"`
- // Edition names the MusicBrainz release the list is from.
- Edition string `json:"edition,omitempty"`
- Tracks []wantTrackView `json:"tracks"`
-}
-
-type wantTrackView struct {
- Disc int32 `json:"disc"`
- Number int32 `json:"number"`
- Title string `json:"title"`
- LengthMs int32 `json:"length_ms"`
- // Missing: the library's file for it is gone.
- Missing bool `json:"missing"`
-}
-
-// handleWantTracks implements GET /api/admin/wanted/{id}/tracks.
-func (h *handlers) handleWantTracks(w http.ResponseWriter, r *http.Request) {
- id, ok := requireURLUUID(w, r, "id")
- if !ok {
- return
- }
- q := dbq.New(h.pool)
- want, err := q.GetWantAlbum(r.Context(), id)
- if errors.Is(err, pgx.ErrNoRows) {
- writeErr(w, apierror.NotFound("want"))
- return
- }
- if err != nil {
- writeErrWithLog(w, h.logger, "admin wanted: tracks", apierror.Internal(err))
- return
- }
- out := wantTracksView{Tracks: []wantTrackView{}}
- if want.AlbumID.Valid {
- rows, err := q.ListLibraryAlbumTracks(r.Context(), want.AlbumID)
- if err != nil {
- writeErrWithLog(w, h.logger, "admin wanted: library tracks", apierror.Internal(err))
- return
- }
- out.Source = "library"
- for _, t := range rows {
- v := wantTrackView{Title: t.Title, LengthMs: t.DurationMs, Missing: t.Missing}
- if t.DiscNumber != nil {
- v.Disc = *t.DiscNumber
- }
- if t.TrackNumber != nil {
- v.Number = *t.TrackNumber
- }
- out.Tracks = append(out.Tracks, v)
- }
- writeJSON(w, http.StatusOK, out)
- return
- }
- eds, err := h.catalogue.Releases(r.Context(), want.ReleaseGroupMbid)
- switch {
- case errors.Is(err, catalogue.ErrNotFound):
- out.Source = "none"
- writeJSON(w, http.StatusOK, out)
- return
- case errors.Is(err, musicbrainz.ErrUnreachable), errors.Is(err, musicbrainz.ErrBadResponse):
- writeErr(w, &apierror.Error{Status: 503, Code: "musicbrainz_unreachable", Message: "MusicBrainz can't be reached", Cause: err})
- return
- case err != nil:
- writeErrWithLog(w, h.logger, "admin wanted: editions", apierror.Internal(err))
- return
- }
- e, ok := standardEdition(eds.Editions)
- if !ok {
- out.Source = "none"
- writeJSON(w, http.StatusOK, out)
- return
- }
- out.Source = "musicbrainz"
- out.Edition = editionLabel(e.Release)
- for _, t := range e.Tracks {
- out.Tracks = append(out.Tracks, wantTrackView{Disc: t.MediumPosition, Number: t.Position, Title: t.Title, LengthMs: t.LengthMs})
- }
- writeJSON(w, http.StatusOK, out)
-}
-
-// standardEdition is the edition a person would call the album: an official
-// one with the fewest tracks, else any with the fewest. The catalogue lists
-// official editions first by date, so a tie goes to the earliest.
-func standardEdition(eds []catalogue.Edition) (catalogue.Edition, bool) {
- best := -1
- for i, e := range eds {
- if len(e.Tracks) == 0 {
- continue
- }
- if best < 0 {
- best = i
- continue
- }
- b := eds[best]
- bo, eo := b.Release.Status == "Official", e.Release.Status == "Official"
- if (eo && !bo) || (eo == bo && len(e.Tracks) < len(b.Tracks)) {
- best = i
- }
- }
- if best < 0 {
- return catalogue.Edition{}, false
- }
- return eds[best], true
-}
-
-// editionLabel names a release for a person: "2023 · CD · GB".
-func editionLabel(r dbq.MbRelease) string {
- var parts []string
- for _, p := range []string{first4(r.ReleaseDate), r.Formats, r.Country} {
- if p != "" {
- parts = append(parts, p)
- }
- }
- return strings.Join(parts, " · ")
-}
-
-func first4(date string) string {
- if len(date) < 4 {
- return ""
- }
- return date[:4]
-}
-
-// handleRemoveWant implements DELETE /api/admin/wanted/{id}.
-func (h *handlers) handleRemoveWant(w http.ResponseWriter, r *http.Request) {
- id, ok := requireURLUUID(w, r, "id")
- if !ok {
- return
- }
- removed, err := h.wanted.RemoveWant(r.Context(), id)
- if err != nil {
- writeErrWithLog(w, h.logger, "admin wanted: remove", apierror.Internal(err))
- return
- }
- if !removed {
- writeErr(w, apierror.NotFound("want"))
- return
- }
- w.WriteHeader(http.StatusNoContent)
-}
-
-// followView is a followed artist.
-type followView struct {
- ArtistMBID string `json:"artist_mbid"`
- ArtistName string `json:"artist_name"`
- ArtistID string `json:"artist_id,omitempty"`
- ProfileID string `json:"profile_id"`
- ProfileName string `json:"profile_name"`
- FollowedAt string `json:"followed_at"`
- CheckedAt *string `json:"checked_at"`
- OpenWants int64 `json:"open_wants"`
- ImageURL string `json:"image_url"`
-}
-
-// handleListFollows implements GET /api/admin/follows.
-func (h *handlers) handleListFollows(w http.ResponseWriter, r *http.Request) {
- rows, err := dbq.New(h.pool).ListFollows(r.Context())
- if err != nil {
- writeErrWithLog(w, h.logger, "admin follows: list", apierror.Internal(err))
- return
- }
- out := make([]followView, 0, len(rows))
- for _, f := range rows {
- v := followView{
- ArtistMBID: f.ArtistMbid, ArtistName: f.ArtistName,
- ProfileID: uuidToString(f.ProfileID), ProfileName: f.ProfileName,
- FollowedAt: formatTimestamp(f.FollowedAt), OpenWants: f.OpenWants,
- ImageURL: artistArtURL(f.ArtistMbid, f.ArtistName),
- }
- if f.ArtistID.Valid {
- v.ArtistID = uuidToString(f.ArtistID)
- }
- if f.CheckedAt.Valid {
- checked := formatTimestamp(f.CheckedAt)
- v.CheckedAt = &checked
- }
- out = append(out, v)
- }
- writeJSON(w, http.StatusOK, out)
-}
-
-// followStateView answers whether one artist is followed, for the control on
-// an artist page.
-type followStateView struct {
- Following bool `json:"following"`
- ProfileID string `json:"profile_id,omitempty"`
- ProfileName string `json:"profile_name,omitempty"`
-}
-
-// handleGetFollow implements GET /api/admin/follows/{mbid}.
-func (h *handlers) handleGetFollow(w http.ResponseWriter, r *http.Request) {
- f, err := dbq.New(h.pool).GetFollow(r.Context(), chi.URLParam(r, "mbid"))
- if errors.Is(err, pgx.ErrNoRows) {
- writeJSON(w, http.StatusOK, followStateView{})
- return
- }
- if err != nil {
- writeErrWithLog(w, h.logger, "admin follows: get", apierror.Internal(err))
- return
- }
- writeJSON(w, http.StatusOK, followStateView{
- Following: true, ProfileID: uuidToString(f.ProfileID), ProfileName: f.ProfileName,
- })
-}
-
-type putFollowBody struct {
- ArtistName string `json:"artist_name"`
- ProfileID string `json:"profile_id"`
- WantExisting bool `json:"want_existing"`
-}
-
-// handlePutFollow implements PUT /api/admin/follows/{mbid}: follow an artist,
-// or change the profile it is followed on. want_existing also wants what the
-// artist has already released that the profile passes and isn't held.
-func (h *handlers) handlePutFollow(w http.ResponseWriter, r *http.Request) {
- admin, ok := requireUser(w, r)
- if !ok {
- return
- }
- var body putFollowBody
- if !decodeBody(w, r, &body) {
- return
- }
- var profile pgtype.UUID
- if body.ProfileID != "" {
- if profile, ok = parseUUID(body.ProfileID); !ok {
- writeErr(w, apierror.BadRequest("bad_profile", "profile_id is not an id"))
- return
- }
- }
- mbid := chi.URLParam(r, "mbid")
- if _, err := h.wanted.Follow(r.Context(), wanted.FollowParams{
- ArtistMBID: mbid, ArtistName: body.ArtistName, ProfileID: profile,
- By: admin.ID, WantExisting: body.WantExisting,
- }); err != nil {
- switch {
- case errors.Is(err, wanted.ErrNotFollowable):
- writeErr(w, apierror.BadRequest("not_followable", "Various Artists and MusicBrainz placeholders can't be followed"))
- case errors.Is(err, wanted.ErrProfileNotFound):
- writeErr(w, apierror.BadRequest("profile_not_found", "no such profile"))
- default:
- writeErrWithLog(w, h.logger, "admin follows: put", apierror.Internal(err))
- }
- return
- }
- h.handleGetFollow(w, r)
-}
-
-// handleDeleteFollow implements DELETE /api/admin/follows/{mbid}. What the
-// follow already wanted stays on the list.
-func (h *handlers) handleDeleteFollow(w http.ResponseWriter, r *http.Request) {
- removed, err := h.wanted.Unfollow(r.Context(), chi.URLParam(r, "mbid"))
- if err != nil {
- writeErrWithLog(w, h.logger, "admin follows: delete", apierror.Internal(err))
- return
- }
- if !removed {
- writeErr(w, apierror.NotFound("follow"))
- return
- }
- w.WriteHeader(http.StatusNoContent)
-}
-
-// releaseProfileView is a release-type profile.
-type releaseProfileView struct {
- ID string `json:"id"`
- Name string `json:"name"`
- PrimaryTypes []string `json:"primary_types"`
- SecondaryTypes []string `json:"secondary_types"`
- IsDefault bool `json:"is_default"`
- FollowCount int64 `json:"follow_count"`
-}
-
-// releaseProfilesResponse also carries MusicBrainz's types, so the editor
-// offers exactly what a profile may hold.
-type releaseProfilesResponse struct {
- Profiles []releaseProfileView `json:"profiles"`
- PrimaryTypes []string `json:"primary_types"`
- SecondaryTypes []string `json:"secondary_types"`
-}
-
-// handleListReleaseProfiles implements GET /api/admin/release-profiles.
-func (h *handlers) handleListReleaseProfiles(w http.ResponseWriter, r *http.Request) {
- rows, err := dbq.New(h.pool).ListReleaseTypeProfiles(r.Context())
- if err != nil {
- writeErrWithLog(w, h.logger, "admin release profiles: list", apierror.Internal(err))
- return
- }
- out := releaseProfilesResponse{
- Profiles: make([]releaseProfileView, 0, len(rows)),
- PrimaryTypes: wanted.PrimaryTypes, SecondaryTypes: wanted.SecondaryTypes,
- }
- for _, p := range rows {
- out.Profiles = append(out.Profiles, releaseProfileView{
- ID: uuidToString(p.ID), Name: p.Name, PrimaryTypes: p.PrimaryTypes,
- SecondaryTypes: p.SecondaryTypes, IsDefault: p.IsDefault, FollowCount: p.FollowCount,
- })
- }
- writeJSON(w, http.StatusOK, out)
-}
-
-type releaseProfileBody struct {
- Name string `json:"name"`
- PrimaryTypes []string `json:"primary_types"`
- SecondaryTypes []string `json:"secondary_types"`
- IsDefault bool `json:"is_default"`
-}
-
-func (b releaseProfileBody) input() wanted.ProfileInput {
- return wanted.ProfileInput{
- Name: b.Name, PrimaryTypes: b.PrimaryTypes, SecondaryTypes: b.SecondaryTypes, IsDefault: b.IsDefault,
- }
-}
-
-// handleCreateReleaseProfile implements POST /api/admin/release-profiles.
-func (h *handlers) handleCreateReleaseProfile(w http.ResponseWriter, r *http.Request) {
- var body releaseProfileBody
- if !decodeBody(w, r, &body) {
- return
- }
- p, err := h.wanted.CreateProfile(r.Context(), body.input())
- if err != nil {
- h.writeProfileErr(w, err)
- return
- }
- writeJSON(w, http.StatusCreated, releaseProfileView{
- ID: uuidToString(p.ID), Name: p.Name, PrimaryTypes: p.PrimaryTypes,
- SecondaryTypes: p.SecondaryTypes, IsDefault: p.IsDefault,
- })
-}
-
-// handleUpdateReleaseProfile implements PUT /api/admin/release-profiles/{id}.
-// A follow on the profile judges new releases by the new types from its next
-// check; releases it has already judged are not judged again.
-func (h *handlers) handleUpdateReleaseProfile(w http.ResponseWriter, r *http.Request) {
- id, ok := requireURLUUID(w, r, "id")
- if !ok {
- return
- }
- var body releaseProfileBody
- if !decodeBody(w, r, &body) {
- return
- }
- p, err := h.wanted.UpdateProfile(r.Context(), id, body.input())
- if err != nil {
- h.writeProfileErr(w, err)
- return
- }
- writeJSON(w, http.StatusOK, releaseProfileView{
- ID: uuidToString(p.ID), Name: p.Name, PrimaryTypes: p.PrimaryTypes,
- SecondaryTypes: p.SecondaryTypes, IsDefault: p.IsDefault,
- })
-}
-
-// handleDeleteReleaseProfile implements DELETE
-// /api/admin/release-profiles/{id}.
-func (h *handlers) handleDeleteReleaseProfile(w http.ResponseWriter, r *http.Request) {
- id, ok := requireURLUUID(w, r, "id")
- if !ok {
- return
- }
- if err := h.wanted.DeleteProfile(r.Context(), id); err != nil {
- h.writeProfileErr(w, err)
- return
- }
- w.WriteHeader(http.StatusNoContent)
-}
-
-func (h *handlers) writeProfileErr(w http.ResponseWriter, err error) {
- switch {
- case errors.Is(err, wanted.ErrProfileInvalid):
- writeErr(w, apierror.BadRequest("profile_invalid", err.Error()))
- case errors.Is(err, wanted.ErrProfileNotFound):
- writeErr(w, apierror.NotFound("profile"))
- case errors.Is(err, wanted.ErrProfileNameUsed):
- writeErr(w, apierror.Conflict("profile_name_taken", "a profile already has that name"))
- case errors.Is(err, wanted.ErrProfileInUse):
- writeErr(w, apierror.Conflict("profile_in_use", "artists are followed on this profile; move them first"))
- case errors.Is(err, wanted.ErrProfileDefault):
- writeErr(w, apierror.Conflict("profile_is_default", "the default profile can't be deleted"))
- default:
- writeErrWithLog(w, h.logger, "admin release profiles", apierror.Internal(err))
- }
-}
-
-// deref is a nullable column's text, or "".
-func deref(s *string) string {
- if s == nil {
- return ""
- }
- return *s
-}
-
-// handleListUnmatched implements GET /api/admin/wanted/unmatched: the
-// Collection's Unmatched section, the albums no MusicBrainz album
-// matched and, after them, the ones a person left off.
-func (h *handlers) handleListUnmatched(w http.ResponseWriter, r *http.Request) {
- rows, err := dbq.New(h.pool).ListUnmatchedLibraryAlbums(r.Context(), unmatchedShown)
- if err != nil {
- writeErrWithLog(w, h.logger, "admin wanted: unmatched", apierror.Internal(err))
- return
- }
- out := struct {
- Items []unmatchedAlbumView `json:"items"`
- }{Items: make([]unmatchedAlbumView, 0, len(rows))}
- for _, u := range rows {
- out.Items = append(out.Items, unmatchedAlbumView{
- AlbumID: uuidToString(u.AlbumID), Title: u.Title, Artist: u.ArtistName, Detail: u.Detail,
- Tracks: u.Tracks, LeftOff: u.Outcome == "dismissed", DecidedAt: formatTimestamp(u.DecidedAt),
- })
- }
- writeJSON(w, http.StatusOK, out)
-}
-
-type manualMatchView struct {
- ReleaseGroupMBID string `json:"release_group_mbid"`
- Title string `json:"title"`
- Artist string `json:"artist"`
- Agree int `json:"agree"`
- Tracks int `json:"tracks"`
-}
-
-// handleMatchUnmatched implements POST /api/admin/wanted/unmatched/{id}/match
-// {"ref": "- Couldn't load indexers and download clients. - -
- {:else if services === null} -Loading indexers and download clients…
- {:else} - -- {#if !canFetch} - No download client is switched on, so nothing can be downloaded. - {:else if !canSearch} - No indexer is switched on, so nothing new can be found. - {:else} - Music can be searched for and downloaded. - {/if} -
- -No indexer yet. Prowlarr is the usual choice.
- {/each} - {#if searchingExchanges.length > 0} -- Also searched: {searchingExchanges.map((s) => s.label).join(', ')}, under download clients. -
- {/if} - {@render addMenu('search', 'Add indexer')} -No download client yet.
- {/each} - {@render addMenu('download', 'Add download client')} -{blurb[service.protocol]}
- -{problem}
- {/if} - {#if result} -- {result.detail} -
- {/if} - -Out-of-library artists drawn from what you've liked and played.
-- {snoozes.length === 0 - ? 'Listen to something or like an artist to start getting suggestions.' - : "Nothing new right now — the artists you've parked are below."} -
- {:else if suggestions.length > 0} -- These come back on their own. Nothing here counts against your taste profile. -
-Loading what the last search found…
- {:else if query.isError} -Couldn't load the releases.
- {:else if items.length === 0} -- Nothing kept from a search yet. Search now, or wait for the schedule. -
- {:else} -- {plural(items.length, 'release', 'releases')} found {relativeTime(items[0].searched_at)}, best first. -
-Reading the blocked releases…
-{:else if blocks.isError && !blocks.isFetchNextPageError} -Couldn't load the blocked releases.
-{:else if items.length === 0} -No releases are blocked.
-- A release you set aside, or one an import found wasn't the album, lands here so it's never - taken again. -
-{b.detail}
- {/if} -- Your library's albums are being added to the catalogue. Albums whose tags don't name them are - matched on MusicBrainz first, by their artist, title and tracks, about one a second. -
-{/if} - -{#if !artists.isPending && (items.length > 0 || q)} -Reading the catalogue…
-{:else if artists.isError && !artists.isFetchNextPageError} -Couldn't load the catalogue.
-{:else if items.length === 0 && q} -- No artist in the catalogue, or album of theirs, is named like '{q}'. -
-{:else if items.length === 0} -The catalogue is empty.
-- Your library's albums land here once it's been scanned, as do the albums users request and - the releases of artists Minstrel follows. -
-- Can't reach the download client {c.client} since {relativeTime(c.since)}: {c.detail}. Its - downloads are left as they are until it answers. -
-{/each} - -{#if downloads.isPending} -Reading the downloads…
-{:else if downloads.isError && !downloads.isFetchNextPageError} -Couldn't load the downloads.
-{:else if items.length === 0} -Nothing has been downloaded yet.
-- A release the search takes, or one you take from a want's Releases, shows here while it - downloads and once it's imported. -
-{condition(d.condition)}
- {/if} - {#if ending(d)} -{ending(d)}
- {/if} - {#if d.import_reason} -- {d.import_reason} -
- {/if} - {#if d.evidence} -{counted(d.evidence)}
- {/if} - {#if filed(d)} -{filed(d)}
- {/if} - {#if seeded(d)} -{seeded(d)}
- {/if} - {#if d.rehomed_from} -Taken for {d.rehomed_from}; it turned out to be this album.
- {/if} -{when(d)}
- - {#if running(d)} -Reading who's followed…
-{:else if follows.isError} -Couldn't load the followed artists.
-{:else if (follows.data ?? []).length === 0} -No artists are followed. Follow one from its artist page.
-{:else} -Reading the unmatched albums…
-{:else if unmatched.isError} -Couldn't load the unmatched albums.
-{:else} - {#if toMatch.length === 0} -Every album in your library is matched.
-- An album no MusicBrainz album matches by its artist, title and tracks lands here, to be - matched by hand or left off. -
-- No MusicBrainz album matched {toMatch.length === 1 ? 'this album' : `these ${toMatch.length} albums`} - by artist, title and tracks, so {toMatch.length === 1 ? "it isn't" : "they aren't"} fetched back if - {toMatch.length === 1 ? 'its' : 'their'} files go missing. Match each to its MusicBrainz album, or - leave it off if MusicBrainz doesn't have it. -
-- Kept in the library, never matched again, and not fetched back if their files go missing. -
-Asking MusicBrainz…
- {:else if search.isError} -Couldn't reach MusicBrainz. A pasted link works the same.
- {:else if asked && results.length === 0} -MusicBrainz has no album like that.
- {:else if results.length > 0} -- {head.held} of {plural(head.albums, 'album', 'albums')} held{head.wanted ? ` · ${head.wanted} wanted` : ''}{head.missing - ? ` · ${head.missing} with files missing` - : ''}{head.downloading ? ` · ${head.downloading} downloading` : ''} -
-Reading the wanted list…
-{:else if wants.isError && !wants.isFetchNextPageError} -Couldn't load {artist ? "this artist's albums" : 'the wanted list'}.
-{:else if items.length === 0 && q} -- Nothing on this list has an artist or album named like '{q}'. -
-{:else if items.length === 0} -- {#if artist}None of this artist's albums are in the catalogue. - {:else if list === 'upgrade'}No album is wanted for a better copy. - {:else}Nothing is wanted.{/if} -
-- {#if list === 'upgrade'} - An album you have below the quality set under - Acquisition, or with a video - rip among its tracks, lands here. - {:else} - A request you approve, a new release from a followed artist, or an album whose files went - missing lands here. - {/if} -
-Reading the tracks…
- {:else if tracks.isError} -- {away ? "MusicBrainz can't be reached, so the tracks can't be shown." : "Couldn't load the tracks."} -
- {:else if tracks.data.source === 'none' || list.length === 0} -MusicBrainz lists no edition of this album, so there are no tracks to show.
- {:else} - {#if tracks.data.source === 'musicbrainz'} -- MusicBrainz's standard edition{tracks.data.edition ? ` · ${tracks.data.edition}` : ''} -
- {/if} -- {dl.running === 0 ? 'Nothing downloading' : `${dl.running} downloading`}. - In the last day, {dl.imported_day} imported{dl.given_up_day ? ` and ${dl.given_up_day} given up` : ''}. - Activity -
- {#if dl.live.length > 0} -
diff --git a/web/src/routes/admin/acquisition/+page.svelte b/web/src/routes/admin/acquisition/+page.svelte
deleted file mode 100644
index 1f306a69..00000000
--- a/web/src/routes/admin/acquisition/+page.svelte
+++ /dev/null
@@ -1,25 +0,0 @@
-
-
-
body
' })); // eslint-disable-next-line @typescript-eslint/no-explicit-any render(AdminLayout, { children } as any); @@ -285,9 +248,7 @@ describe('admin layout', () => { ]); expect(links).toEqual([ ['/admin', 'Overview'], - ['/admin/requests', 'Requests 2'], ['/admin/library', 'Library'], - ['/admin/acquisition', 'Acquisition'], ['/admin/metadata', 'Metadata sources'], ['/admin/email', 'Email & network'], ['/admin/users', 'Users'], diff --git a/web/src/routes/admin/library/+page.svelte b/web/src/routes/admin/library/+page.svelte index 657e0dc8..e7b26da1 100644 --- a/web/src/routes/admin/library/+page.svelte +++ b/web/src/routes/admin/library/+page.svelte @@ -147,13 +147,7 @@- Every artist and album in your collection: what the library holds, and what Minstrel is - getting for it, from user requests, new releases of the artists it follows, and albums whose - files went missing. It searches your indexers on a schedule, checks each download against - MusicBrainz, and files what's right into the library. Searching and quality are set under - Acquisition. + What in the library needs looking at: tracks people reported, songs held twice, files from + doubtful sources, and files that went missing.
- {#if verdict} -- {verdict} - {#if counts?.attention} - See Activity - {:else if counts?.unmatched} - See Unmatched - {/if} -
- {/if} - {#if health.length > 0} -- To look at: - {#each health as d, i (d.key)}{queueLabel(d, countOf(summary, d.key))}{i < health.length - 1 ? ', ' : '.'} - {/each} -
+ {#if summary} + {#if waiting.length > 0} ++ To look at: + {#each waiting as d, i (d.key)}{queueLabel(d, countOf(summary, d.key))}{i < waiting.length - 1 ? ', ' : '.'} + {/each} +
+ {:else} +Nothing needs you.
+ {/if} {/if}Reading the queues…
+ {:else if summaryQ.isError} +
+ Couldn't read the queues.
+
+ No {clear.map((d) => d.none).join(', ')}. +
+ {/if} + {/if} +body
` })); afterEach(() => vi.clearAllMocks()); const none = { count: 0, since: null }; -const clear: AdminSummary = { requests: none, reports: none, duplicates: none, suspect: none, missing: none }; +const clear: AdminSummary = { reports: none, duplicates: none, suspect: none, missing: none }; -function setup(c: WantCounts = counts, path = '/collection', summary: AdminSummary = clear) { - state.pageUrl = new URL(`http://localhost${path}`); +function mockSummary(summary: AdminSummary) { vi.mocked(createAdminSummaryQuery).mockReturnValue( mockQuery({ data: summary }) as unknown as ReturnTypeTracks that hold the same recording more than once. Most settle on their own: a song held twice on one album is merged, copies on different releases count as one song, two versions of a song - are left alone, and a file filling another song's track slot is removed so the album is fetched - again. What's left here needs you. + are left alone, and a file filling another song's track slot is removed, leaving that slot + missing. What's left here needs you.
@@ -344,12 +339,12 @@ {:else if item.action === 'duplicate_wrong_file' && !item.details.kept_title}{item.details.artist}: one recording was filed as {removed.map((r) => r.title).join(' and ')}, - and it is none of them. Removed; the album will be fetched again. + and it is none of them. Removed; those slots show as missing.
{:else if item.action === 'duplicate_wrong_file'}{item.details.artist}: the recording of {item.details.kept_title} was also filed as - {removed.map((r) => r.title).join(', ')}. Removed from there; the album will be fetched again. + {removed.map((r) => r.title).join(', ')}. Removed from there; those slots show as missing.
{:else}@@ -455,11 +450,11 @@ {#if refetching === group.id} Removing… {:else if confirmingRefetch === group.id} - Remove {plural(refetchRemoves(group).length, 'file', 'files')} and get it again + Remove {plural(refetchRemoves(group).length, 'file', 'files')} {:else if picked} Keep “{picked.title}”… {:else} - Get it again… + Remove the copies… {/if} {:else} @@ -527,8 +522,8 @@ {:else} All {plural(group.members.length, 'copy', 'copies')} will be deleted from disk. {/if} - Their tracks stay, marked missing, with their likes and plays. The release the files came - from is set aside, and the album is fetched again from another one. + Their tracks stay, marked missing, with their likes and plays, until a good copy is in the + library folder.
Tracks whose audio file the last scan couldn't find. They keep their play - history and likes, and stop being offered anywhere until the file returns. + history and likes, and stop being offered anywhere until a file is back in the library + folder.
-- {#if policy} - {policy.enabled - ? `Albums missing for ${policy.grace_hours} hours go back on the wanted list by themselves.` - : 'Missing albums are not wanted back automatically.'} - {/if} - - {policy ? 'Change' : 'Set what happens to missing files'} - -
- {#if query.isPending}Checking what's missing…
{:else if query.isError} @@ -158,72 +84,6 @@- {#if r.held_at} - Held again {relativeTime(r.held_at)}. The rows here clear on the next scan. - {:else if r.wanted_at} - Wanted back since {relativeTime(r.wanted_at)}. - {/if} -
- {#if w && !r.held_at} -- {wantStatus(w)} - {#if needsYou(w)} - Decide on Activity - {/if} -
- {/if} -- Search MusicBrainz to add new artists, albums, or tracks. -
-Searching…
- {:else if results.length === 0} -Nothing to add for that search yet.
- {:else} -- Requesting {modalResult.name} - will add {modalResult.secondary_text}. - Continue? -
-- What you've asked Minstrel to add to the library. -
-Reading the ledger…
- {:else if requests.length === 0} -Nothing requested yet.
- {:else} -