Compare commits

..
29 Commits
Author SHA1 Message Date
bvandeusenandClaude Opus 5.5 dabe1e052d feat: genre mixes pick specific genres that don't share songs (#6933)
release / go (push) Successful in 2m0s
release / web (push) Successful in 1m33s
release / govulncheck (push) Successful in 18s
release / integration (push) Successful in 5m11s
release / android (push) Successful in 4m14s
release / Build signed APK (push) Successful in 4m38s
release / Attach APK to the Release (tag releases only) (push) Skipped
release / Build + push container image (push) Successful in 29s
release / Verify release artifacts (tag releases only) (push) Skipped
The shelf had four mixes built largely from the same lo-fi beats: about
1,250 tracks carry Hip Hop, Lo-Fi, Instrumental, Downtempo and four more
tags at once, and the near-duplicate guard compared artists, not songs, so
each tag passed as its own channel.

Genre selection now works on songs:
- ListGenreMixCandidates returns each genre's track ids instead of its
  artist ids.
- A broad genre, one whose songs mostly also carry clearly narrower
  candidates (Rock over Pop Punk and Ska Punk), is considered after the
  specific genres of its tier.
- A genre is kept only when at least half its songs are not already in a
  kept genre.
- Each mix leaves out songs tagged with a genre kept before it
  (ListGenreMixTracks exclude_genres), so a song is in one mix at most.

The tiers, the minimum shelf and the minimum mix length are unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-07 14:55:19 -05:00
bvandeusenandClaude Opus 5.5 14debe2ea9 feat: genre mixes lean toward what the listener likes (#6921)
release / web (push) Successful in 1m40s
release / go (push) Successful in 2m5s
release / govulncheck (push) Successful in 19s
release / integration (push) Successful in 5m2s
release / android (push) Successful in 4m40s
release / Build signed APK (push) Successful in 5m7s
release / Attach APK to the Release (tag releases only) (push) Skipped
release / Build + push container image (push) Successful in 32s
release / Verify release artifacts (tag releases only) (push) Skipped
Within a genre both halves of the mix were a uniform daily draw: a song
played forty times was no likelier than one played once, and the unheard
40% ignored who the artist was. ListGenreMixTracks now draws each half by a
date-seeded weighted lottery (Efraimidis-Spirakis: smallest -ln(u)/weight
wins), so the mix still changes daily and holds within a day.

- Familiar: 1 + ln(1 + plays) + 1 if liked, times the share of plays that
  weren't skips.
- Unheard: 1 + ln(1 + the artist's plays + 2 x its liked tracks)
  + 2 x the artist's best similarity to the user's own artists, times the
  artist's non-skip share. An unknown artist weighs 1, so new artists still
  come up.

Which genres are offered, the exclusions and the familiar/unheard blend
are unchanged. DB tests check the lean averaged over ten days, and that a
stranger's tracks stay in the pool.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-06 14:21:04 -05:00
bvandeusenandClaude Opus 5.5 adbb3a3cd8 fix(android): the launch profile refresh waits for the stored login
release / govulncheck (push) Successful in 18s
release / Build signed APK (push) Successful in 4m37s
release / web (push) Successful in 1m25s
release / go (push) Successful in 1m50s
release / integration (push) Successful in 4m36s
release / android (push) Successful in 3m59s
release / Attach APK to the Release (tag releases only) (push) Skipped
release / Build + push container image (push) Successful in 15s
release / Verify release artifacts (tag releases only) (push) Skipped
AuthController refreshes /api/me when the app starts, to pick up account
flags such as debug mode. That ran before the session cookie had loaded
from the Keystore, read the user as signed out, and skipped. It now waits
for the load (bounded by the hydration deadline). The diagnostics
uploader's drain waits the same way, so a launch-time trigger isn't
skipped. BootReceiver already waited.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-06 11:55:59 -05:00
bvandeusenandClaude Opus 5.5 e73950458f test(android): the slow-hydration test's row carries the mock server URL
release / web (push) Successful in 1m34s
release / govulncheck (push) Successful in 20s
release / integration (push) Successful in 4m57s
release / go (push) Successful in 1m58s
release / android (push) Successful in 4m30s
release / Build signed APK (push) Successful in 4m56s
release / Attach APK to the Release (tag releases only) (push) Skipped
release / Build + push container image (push) Successful in 16s
release / Verify release artifacts (tag releases only) (push) Skipped
On a real background dispatcher the store's row collector lands after
construction and reset the base URL to the default, so the request went to
localhost:8080 instead of the mock server. The row now holds the server URL.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-06 10:30:42 -05:00
bvandeusenandClaude Opus 5.5 454d71d079 fix(android): the app stops signing itself out at launch (#6911)
release / web (push) Successful in 1m24s
release / go (push) Successful in 1m52s
release / govulncheck (push) Successful in 18s
release / integration (push) Successful in 4m43s
release / android (push) Failing after 3m25s
release / Build signed APK (push) Successful in 4m41s
release / Attach APK to the Release (tag releases only) (push) Skipped
release / Build + push container image (push) Skipped
release / Verify release artifacts (tag releases only) (push) Skipped
At launch the stored session cookie loads from the Keystore off the main
thread. A request sent before that finished went out without it, typically
the library sync that fires as soon as the server answers. The server then
answered 401, and the interceptor cleared the store on any 401, which also
stopped the load from restoring the cookie and wrote null over the stored
one. The user had to sign in on every launch. A session lost mid-use also
left them on Home with every request failing, because the app only routes
to sign-in when it starts.

- The interceptor waits for the stored cookie to load before attaching it,
  bounded by the existing hydration deadline.
- A 401 signs out only when the request carried the current cookie
  (compare-and-clear). A request sent without one, or with a cookie a newer
  sign-in has replaced, leaves the login alone.
- When the server refuses the session mid-use, the app opens the sign-in
  screen. A sign-out from Settings does not trigger this.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-06 10:22:59 -05:00
bvandeusenandClaude Opus 5.5 8251fef604 test: seedGenre gives each artist one artist row and one album
release / web (push) Successful in 1m25s
release / go (push) Successful in 1m49s
release / govulncheck (push) Successful in 22s
release / android (push) Successful in 4m22s
release / Build + push container image (push) Successful in 1m12s
release / Verify release artifacts (tag releases only) (push) Skipped
release / integration (push) Successful in 4m52s
release / Build signed APK (push) Successful in 4m46s
release / Attach APK to the Release (tag releases only) (push) Skipped
seedTrack mints a new artist per call when there is no mbid, so the genre
fixtures held one artist per track and the candidate query's artist_ids
counted 30 where the test meant 6.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-05 22:39:29 -05:00
bvandeusenandClaude Opus 5.5 dbaa8b7d57 feat: genre mixes — a daily playlist per genre you listen to, on a "Your genres" Home row (#6904)
release / Build signed APK (push) Successful in 4m34s
release / Attach APK to the Release (tag releases only) (push) Skipped
release / Build + push container image (push) Skipped
release / Verify release artifacts (tag releases only) (push) Skipped
release / web (push) Successful in 1m24s
release / govulncheck (push) Successful in 25s
release / go (push) Successful in 1m47s
release / integration (push) Failing after 4m38s
release / android (push) Successful in 4m3s
A new system playlist kind, genre_mix: up to 10 "{Genre} mix" playlists per
user, rebuilt daily with the other system mixes.

Which genres: library genres with at least 25 playable tracks across 3
artists, ranked in tiers — taste-profile genres first (matched
case-insensitively, so ID3 "Post-Punk" and folksonomy "post-punk" are one),
then genres the user has played or liked, then the rest of the library, which
only tops a short shelf up to 4. A genre whose artist set is a near
duplicate of one already picked (Jaccard >= 0.5) is skipped, so "Lo-Fi" and
"Lo-Fi Hip Hop" don't both appear but "Alternative Rock" survives "Rock". A
genre that can't fill 15 tracks gives way to the next.

Each mix is 50 tracks: 3 in 5 the user already knows (played through or
liked), the rest unheard in that genre, interleaved, with the usual
per-album/per-artist caps. Tracks skipped twice and more often than played
are left out. The order changes daily and holds within a day. Picks are
stamped taste / taste_unheard, and genre_mix plays count toward rotation and
the recommendation metrics.

Migration 0102 adds genre_mix to the playlists variant constraints.

Web and Android each get a "Your genres" row in name order, hidden until a
mix exists. Kinds with several playlists per user now play by playlist id
wherever the client used to test seed_artist_id (web card, detail page
self-heal), keyed on the server's refreshable flag; on Android genre_mix
joins songs_like_artist as not refreshable. The web detail page's
"mix was rebuilt" reload now finds a Songs-like or genre mix by name instead
of calling the by-kind shuffle, which those kinds don't have.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-05 22:30:41 -05:00
bvandeusenandClaude Opus 5.5 477f453100 feat(android): settings read as a menu — every group on one Section plate, every setting one SettingRow (#6768)
release / web (push) Successful in 1m19s
release / govulncheck (push) Successful in 18s
release / android (push) Successful in 4m16s
release / Build signed APK (push) Successful in 4m38s
release / go (push) Successful in 1m48s
release / integration (push) Successful in 4m29s
release / Attach APK to the Release (tag releases only) (push) Skipped
release / Build + push container image (push) Successful in 14s
release / Verify release artifacts (tag releases only) (push) Skipped
Settings and Notification settings were hand-rolled ElevatedCards, each with
its own padding, title style and row layout, and several rows carried
two- and three-line explanations. They are now built from one vocabulary in
theme/Section.kt: Section (the plate), SettingRow (title, one short line, an
optional icon, the control at the end), SwitchRow and ChoiceRow (the whole
row toggles or opens the choices), RowValue, RowSpinner, RowChevron, Muted.

The plate is Minstrel's own raised surface with its radius tokens, so the
app keeps its look. Sign out, sync and clear cache are rows, not loose
buttons; the theme, leveling, loudness target and storage caps are choice
rows; the About card (also the update sheet on Home) shows the version as a
value and the update as a row with its one button.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-02 22:31:25 -05:00
bvandeusenandClaude Opus 5.5 598abeb8cf fix: a fresh app install syncs the whole library, not the last 30 days of changes (#6778)
release / govulncheck (push) Successful in 14s
release / web (push) Successful in 1m20s
release / go (push) Successful in 1m44s
release / android (push) Successful in 4m54s
release / Build signed APK (push) Successful in 5m10s
release / integration (push) Successful in 5m19s
release / Build + push container image (push) Successful in 15s
release / Attach APK to the Release (tag releases only) (push) Skipped
release / Verify release artifacts (tag releases only) (push) Skipped
The change feed dropped every row older than 30 days, and an app syncing
from cursor 0 builds its library from the feed, so a fresh install, cleared
data or a 410 reset got only what had changed lately.

The compactor now keeps each entity's latest change however old. It drops a
row once a newer one for the same entity exists, and a delete once it is 30
days old, raising a stored mark (library_changes_trimmed) past it; an app
whose cursor is below the mark is sent back to 0, as before. Migration 0101
adds the index the compactor needs, seeds the mark where the old compactor
left off, and gives every artist, album and track whose latest row is not an
upsert a fresh one, so the rows the old compactor dropped come back. The
backup carries the mark with the feed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-02 22:07:42 -05:00
bvandeusenandClaude Opus 5.5 ad48d8c8c9 fix: the route-registration test passes Mount its restart hook
release / govulncheck (push) Successful in 16s
release / web (push) Successful in 1m25s
release / android (push) Successful in 4m57s
release / Build signed APK (push) Successful in 5m9s
release / go (push) Successful in 1m43s
release / integration (push) Successful in 5m21s
release / Attach APK to the Release (tag releases only) (push) Skipped
release / Build + push container image (push) Successful in 22s
release / Verify release artifacts (tag releases only) (push) Skipped
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-02 20:25:39 -05:00
bvandeusenandClaude Opus 5.5 68e1e47534 fix: a new server takes a backup after its first scan, and restarts to load the restored settings (#6786)
release / Attach APK to the Release (tag releases only) (push) Canceled after 0s
release / Build + push container image (push) Canceled after 0s
release / Verify release artifacts (tag releases only) (push) Canceled after 0s
release / govulncheck (push) Successful in 16s
release / go (push) Failing after 49s
release / web (push) Successful in 1m9s
release / integration (push) Failing after 4m14s
release / android (push) Canceled after 4m38s
release / Build signed APK (push) Canceled after 4m38s
A one-off end-to-end run of the :dev image (scratch Postgres, generated
FLAC files, no outbound network) found that the web restore, done the way
docs/hosting.md describes, was refused with not_empty: a new server's
startup scan fills the library before anyone signs up, and Restore counted
tracks as occupied. The command-line restore passed the same run.

- Restore now asks only that the instance has no accounts. The library the
  first scan found is replaced by the backup's (every foreign key into
  artists/albums/tracks cascades or sets null), and the restore locks the
  library tables so a running scan waits rather than adding to them.
- Settings are read once at boot by about ten services, the library folders
  among them, so after a web restore the server ran on its defaults until
  restarted. The restore endpoint now answers, then asks the process to
  restart in place (syscall.Exec of the same binary, arguments and
  environment), which needs no container restart policy. The startup scan
  of the restarted server replaces the scan the endpoint used to start.
- The restore form says Minstrel is restarting; the docs say a scanned new
  server still takes a backup.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-02 20:21:00 -05:00
bvandeusenandClaude Opus 5.5 a521326686 fix: the restore form's file picker isn't required, so the form submits once a file is chosen
release / govulncheck (push) Successful in 17s
release / web (push) Successful in 1m24s
release / go (push) Successful in 1m40s
release / android (push) Successful in 5m12s
release / integration (push) Successful in 5m24s
release / Build signed APK (push) Successful in 5m35s
release / Attach APK to the Release (tag releases only) (push) Skipped
release / Build + push container image (push) Successful in 23s
release / Verify release artifacts (tag releases only) (push) Skipped
The Restore button already waits for a file. A required file input also kept
jsdom from submitting after a test picked one.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-02 16:16:27 -05:00
bvandeusenandClaude Opus 5.5 29042e4eb2 feat: a new Minstrel restores a backup, from the web app or the command line, then rescans (#6775)
release / govulncheck (push) Successful in 29s
release / web (push) Failing after 1m21s
release / go (push) Successful in 1m39s
release / integration (push) Successful in 5m26s
release / android (push) Successful in 5m27s
release / Build signed APK (push) Successful in 5m44s
release / Attach APK to the Release (tag releases only) (push) Skipped
release / Build + push container image (push) Skipped
release / Verify release artifacts (tag releases only) (push) Skipped
A new server (no accounts, no library) offers "Restore a backup" beside
"Create account". The file goes up as the request body to
POST /api/auth/restore with the setup token from the log in X-Setup-Token,
since the backup's admins become this server's. It loads in one transaction,
then a library scan starts and finds each file at its path. The same load runs
offline as `minstrel restore FILE`, which migrates the schema first.

That path alone gets a 1 GiB, 15-minute body allowance instead of 4 MiB and
30s, and skips the JSON-only check for cookie writes: a stale cookie from the
old server would otherwise refuse the upload, and the custom header already
rules out a cross-site form.

The library sync answers 410 cursor_ahead to a cursor past the last one the
server has handed out, read from the change sequence, so an app synced with
the old server after the backup was made starts over instead of skipping the
ids the restored server reuses. Restore now leaves each sequence as having
issued the restored maximum, so that reading holds; a compacted log still
counts its handed-out cursors.

backup.Read keeps the underlying error in its chain, so an oversized upload
answers 413 rather than "not a backup". The hosting docs say how to put the
music back first and what is entered again afterwards.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-02 16:10:06 -05:00
bvandeusenandClaude Opus 5.5 5090c89d20 fix: the backup panel test reads the music sizes as formatBytes prints them
release / govulncheck (push) Successful in 14s
release / web (push) Successful in 1m23s
release / go (push) Successful in 1m40s
release / android (push) Successful in 5m4s
release / Build signed APK (push) Successful in 5m15s
release / integration (push) Successful in 5m25s
release / Attach APK to the Release (tag releases only) (push) Skipped
release / Build + push container image (push) Successful in 22s
release / Verify release artifacts (tag releases only) (push) Skipped
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-02 15:35:58 -05:00
bvandeusenandClaude Opus 5.5 967763f6ce feat: the backup can include the music, sending each library file once to music/ and again only when it changes (#6774)
release / govulncheck (push) Successful in 15s
release / web (push) Failing after 1m17s
release / go (push) Successful in 1m33s
release / integration (push) Successful in 5m16s
release / Build signed APK (push) Successful in 6m13s
release / Attach APK to the Release (tag releases only) (push) Skipped
release / Build + push container image (push) Skipped
release / Verify release artifacts (tag releases only) (push) Skipped
release / android (push) Successful in 5m56s
With "Include the music" on, each night's run sends the library files the
destination doesn't hold yet, after the database file has landed, for up to
six hours; a large library spans several nights the first time. Files keep
their path under the library folders' common root, beside database/ in the
same folder or bucket.

backup_music_files records each file's size and mtime as sent, so a night
sends only new and changed files, and a rescan that touches a track but finds
the same file sends nothing. A changed file goes to its old name; Minstrel
never deletes there. An object the bucket refuses (a locked overwrite) is
kept with its answer and goes again once the file changes or the settings are
saved; three refusals in a row stop the night's music, since that is a key
that may not write under music/. A folder copy is written under a .part name
and renamed, keeping the file's time.

The panel shows how much of the library is in, the refused files, and each
run's music count and note. The verdict still turns on the database file.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-02 15:26:20 -05:00
bvandeusenandClaude Opus 5.5 ff5ee755f2 fix: the backup test result says its count through plural(), and the panel tests read the size and folder field as shown
release / govulncheck (push) Successful in 17s
release / web (push) Successful in 1m30s
release / go (push) Successful in 1m46s
release / android (push) Successful in 5m23s
release / integration (push) Successful in 5m32s
release / Build signed APK (push) Successful in 5m33s
release / Attach APK to the Release (tag releases only) (push) Skipped
release / Build + push container image (push) Successful in 24s
release / Verify release artifacts (tag releases only) (push) Skipped
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-02 15:04:32 -05:00
bvandeusenandClaude Opus 5.5 f7222732e3 fix: the route test passes the backup service, and the backup panel narrows its status inside each column
release / govulncheck (push) Successful in 19s
release / web (push) Failing after 1m32s
release / go (push) Successful in 1m47s
release / android (push) Successful in 5m16s
release / integration (push) Successful in 5m31s
release / Build signed APK (push) Successful in 5m24s
release / Attach APK to the Release (tag releases only) (push) Skipped
release / Build + push container image (push) Skipped
release / Verify release artifacts (tag releases only) (push) Skipped
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-02 14:56:26 -05:00
bvandeusenandClaude Opus 5.5 9d324ab2d0 feat: Minstrel backs itself up nightly to a folder or an S3 bucket, off until an admin switches it on (#6773)
release / integration (push) Failing after 3m29s
release / web (push) Failing after 15s
release / govulncheck (push) Successful in 16s
release / go (push) Failing after 41s
release / android (push) Successful in 4m7s
release / Attach APK to the Release (tag releases only) (push) Canceled after 0s
release / Build + push container image (push) Canceled after 0s
release / Verify release artifacts (tag releases only) (push) Canceled after 0s
release / Build signed APK (push) Canceled after 5m34s
Admin › Backups shows a verdict first (off, not set up, waiting, backed up,
overdue, failing since when), Back up now, Test destination and the latest
runs, with the settings beside them.

- backup.Service: one run at a time, bounded at 30 minutes; the schedule
  ticks every 5 minutes, catches up a slot missed while the server was down,
  and retries a failure after an hour and then after as long as it has been
  failing. Runs are recorded in backup_runs, pruned after 90 days; a run cut
  off by a shutdown is marked failed at the next start.
- A folder destination writes database/minstrel-<UTC stamp>.tar and refuses
  a folder that isn't there. An S3 destination stages the file, hashes it,
  and uploads it under database/ through a throttled reader (4 MiB/s by
  default, 0 for no limit).
- Test destination checks the form as typed and only lists.
- The bucket secret is sealed like every stored credential; backup_settings
  and backup_runs are left out of the backup itself.
- The panel polls only while a run is going, capped at the run limit plus
  five minutes.

The music switch named in the task is left to #6774, so no control ships
that does nothing.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-02 14:50:55 -05:00
bvandeusenandClaude Opus 5.5 faf19ef67b test: the file-name test's later backup is later in UTC too
release / govulncheck (push) Successful in 22s
release / web (push) Successful in 1m33s
release / go (push) Successful in 1m49s
release / android (push) Successful in 4m59s
release / integration (push) Successful in 5m27s
release / Build signed APK (push) Successful in 5m1s
release / Attach APK to the Release (tag releases only) (push) Skipped
release / Build + push container image (push) Successful in 14s
release / Verify release artifacts (tag releases only) (push) Skipped
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-02 14:01:31 -05:00
bvandeusenandClaude Opus 5.5 e11011e977 fix: the backup package checks or explicitly drops every error the linter flags
release / govulncheck (push) Successful in 24s
release / go (push) Failing after 1m42s
release / web (push) Successful in 1m25s
release / android (push) Successful in 4m52s
release / Build signed APK (push) Successful in 5m1s
release / integration (push) Canceled after 5m17s
release / Attach APK to the Release (tag releases only) (push) Canceled after 0s
release / Build + push container image (push) Canceled after 0s
release / Verify release artifacts (tag releases only) (push) Canceled after 0s
Closing a refused S3 response and a test file, and the fake bucket's
writes, now say they ignore the error on purpose; handler parameters the
tests don't use are named _.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-02 13:56:22 -05:00
bvandeusenandClaude Opus 5.5 b6ae205862 feat: Minstrel can put and list objects in a path-style S3 bucket, signed with SigV4 and no SDK (#6771)
release / govulncheck (push) Successful in 14s
release / android (push) Successful in 4m37s
release / Build signed APK (push) Successful in 4m47s
release / integration (push) Failing after 5m3s
release / Attach APK to the Release (tag releases only) (push) Skipped
release / Build + push container image (push) Skipped
release / Verify release artifacts (tag releases only) (push) Skipped
release / go (push) Failing after 1m7s
release / web (push) Successful in 1m14s
Bucket.Put stores a body under a key, signed over its SHA-256; Bucket.List
pages through ListObjectsV2. A backup only adds and lists, so an
append-only key is enough. Keys are encoded once, exactly as signed, so
music paths with spaces, plus signs and accents reach the bucket intact.
A refusal names S3's code in words ("the secret key is wrong").

The signer is pinned to AWS's published GET Object, PUT Object and
ListObjects examples.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-02 13:51:02 -05:00
bvandeusenandClaude Opus 5.5 8be69b752a feat: a backup is one uncompressed tar of per-table JSON Lines that stays the same bytes day to day (#6769)
The file is minstrel-<UTC stamp>.tar: manifest.json first, then
tables/<table>.jsonl, rows one per line in primary-key order. The run's
time lives only in the manifest and member headers are fixed, so a
deduplicating off-site copy stores unchanged tables once.

WriteFile writes to a hidden .part file, syncs it and renames it, so a
watched folder never shows half a backup. Read refuses anything that
isn't a whole backup: another format, a newer format version, a cut-off
file, or a member holding fewer rows than the manifest says.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-02 13:51:02 -05:00
bvandeusenandClaude Opus 5.5 a67897a6cd feat: Minstrel can read its irreplaceable data out as JSON Lines and load it back into an empty instance (#6770)
release / web (push) Successful in 1m17s
release / android (push) Successful in 4m41s
release / Build signed APK (push) Successful in 4m52s
release / govulncheck (push) Successful in 15s
release / go (push) Successful in 1m29s
release / integration (push) Successful in 5m3s
release / Attach APK to the Release (tag releases only) (push) Skipped
release / Build + push container image (push) Successful in 14s
release / Verify release artifacts (tag releases only) (push) Skipped
internal/backup names every table a backup holds, in parent-first order,
and every table it leaves out with the reason: what the server computes
from the music files, provider caches it fetches again, sign-ins and
one-time links, and logs of past runs. The library's identity rows travel
because every like, play and playlist entry hangs off a track id, and the
sync change feed travels because the apps' first sync reads it.

Sealed credentials are left out of the file, and so are the markers that
say tag and art enrichment are done, so a restored server asks for the
credentials again and redoes that work rather than believing it has it.

Snapshot reads one REPEATABLE READ, read-only transaction, each table in
primary-key order, keys sorted. Restore refuses an instance with accounts
or tracks and a backup from a newer schema, replaces seeded settings,
inserts by column intersection in batches, skips generated columns, and
carries serial sequences on past the restored ids.

Guards: every schema table is backed up or excluded; every backed-up
table comes after what it references and references nothing excluded;
every omitted column exists; a restore gives back the backup byte for byte.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-02 13:43:16 -05:00
bvandeusenandClaude Opus 5.5 a8a2d20307 feat: stored credentials are sealed at rest, so a copy of the database alone reveals none of them (#6767)
release / govulncheck (push) Successful in 28s
release / web (push) Successful in 1m18s
release / go (push) Successful in 1m35s
release / android (push) Successful in 4m45s
release / Build signed APK (push) Successful in 4m57s
release / integration (push) Successful in 5m3s
release / Attach APK to the Release (tag releases only) (push) Skipped
release / Build + push container image (push) Successful in 1m14s
release / Verify release artifacts (tag releases only) (push) Skipped
Minstrel keeps credentials it has to read back: indexer, Prowlarr and
download-client keys and passwords, the AcoustID key, cover-art and tag
provider keys, the SMTP password, and each user's ListenBrainz token and
Subsonic password. They were stored in the clear; they are now sealed with
AES-256-GCM under a per-column HKDF key derived from the stream secret, each
value bound to its row.

- internal/secret: Seal/Open/Reveal, and Columns, the one list of sealed
  columns (a backup will leave them out). Install takes the master key at
  startup; SealStored seals any value still in the clear, so an existing
  database is brought over in place on the first boot.
- Every write seals and every read opens: acquire (one read path, getService
  / usableServices), acoustid settings, SMTP, ListenBrainz, Subsonic auth and
  password generation, cover-art and tag provider settings.
- A value that does not open (the stream secret changed) is logged and reads
  as unset, to be entered again in Settings. docs/security.md says so.
- Tests: secret unit tests (wrong owner/purpose/key, malformed, no key), a
  SealStored integration test over all eight columns plus a schema check that
  each listed column exists, and the existing tests now assert sealed storage.
  dbtest installs a test key for integration tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-02 13:23:19 -05:00
bvandeusenandClaude Opus 5.5 be642cd53f ci: main builds and bundles its own signed APK, so a merge ships the app without a release (#6465)
release / web (push) Successful in 1m51s
release / go (push) Successful in 2m6s
release / govulncheck (push) Successful in 20s
release / integration (push) Successful in 6m53s
release / android (push) Successful in 5m41s
release / Build signed APK (push) Successful in 5m38s
release / Attach APK to the Release (tag releases only) (push) Skipped
release / Build + push container image (push) Successful in 18s
release / Verify release artifacts (tag releases only) (push) Skipped
android-release now runs on main as well as dev and tags, and image-release
requires its success on every ref. :latest carries the stable APK built from
main's own commit instead of the newest release's, so a merged app change
reaches stable phones on the next pull. Releases stay as optional bookmarks
that attach the APK for download.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-26 12:49:39 -04:00
bvandeusenandClaude Opus 5.5 7d5dcd454e docs: two comments state the rule rather than its history (#6277)
release / govulncheck (push) Successful in 17s
release / web (push) Successful in 1m47s
release / go (push) Successful in 2m2s
release / android (push) Successful in 5m49s
release / Build signed APK (releases and dev) (push) Successful in 6m1s
release / integration (push) Successful in 6m18s
release / Attach APK to the Release (tag releases only) (push) Skipped
release / Build + push container image (push) Successful in 18s
release / Verify release artifacts (tag releases only) (push) Skipped
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-26 11:20:17 -04:00
bvandeusenandClaude Opus 5.5 3fa5d9fc3a fix: an app update is offered only when the app changed; casting no longer sticks on "loading" when the same queue plays again (#6455, #6456)
release / govulncheck (push) Successful in 16s
release / web (push) Successful in 1m30s
release / go (push) Successful in 1m46s
release / android (push) Successful in 5m2s
release / Build signed APK (releases and dev) (push) Successful in 5m13s
release / integration (push) Successful in 5m22s
release / Attach APK to the Release (tag releases only) (push) Skipped
release / Build + push container image (push) Successful in 13s
release / Verify release artifacts (tag releases only) (push) Skipped
ci/version.sh --apk names the APK from android/ alone, and the APK lane
calls it. The app offers an update when the server's APK name differs from
its own, so a name derived from the whole repo offered every phone a rebuild
of the app it already had on each server-only deploy. The image keeps the
union set: it bundles the APK under client/.

Casting (#6456):
- setMediaItems with the tracks already queued (play the queued album from
  another track) cleared the cast and waited for a resync that the id-keyed
  collector never ran. It is now a move within the queue: Seek TRACK_NR on a
  Sonos, start that track on a one-track renderer; the play() goes through.
- A queue replaced through setMediaItems could take the Sonos incremental
  diff, which cleared target without restoring active, so the app thought it
  was back on the phone while the speaker played on, and the chosen start
  track was ignored. The diff now runs only while the route is still active;
  a replaced queue reloads in full at its start index.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-26 10:08:02 -04:00
bvandeusenandClaude Opus 5.5 fa75bd08d7 feat: a newer app is told in a snackbar on Home and installed from a sheet holding Settings' card, fetched ahead on Wi-Fi (#6452)
release / govulncheck (push) Successful in 21s
release / web (push) Successful in 1m41s
release / go (push) Successful in 1m55s
release / android (push) Successful in 5m19s
release / Build signed APK (releases and dev) (push) Successful in 5m31s
release / integration (push) Successful in 5m39s
release / Attach APK to the Release (tag releases only) (push) Skipped
release / Build + push container image (push) Successful in 28s
release / Verify release artifacts (tag releases only) (push) Skipped
Mirrors Librarian (#6446, #6448), after Gallery's update tray (canon #6367).

- Home shows "Version X of the app is on your server." / "…is downloaded
  and ready." with Update / Install; the button opens a bottom sheet
  holding the same About card Settings shows, so there is one install
  path. Each build is told at most once a day, kept on disk.
- The shell-wide update banner and its in-memory dismissals are gone.
- AppUpdates (singleton) holds the offer and the install, so Home and
  Settings show the same state. Checks run when either screen opens, on
  every return to the front, every 5 minutes while shown, and on Home's
  pull to refresh.
- UpdateFetchWorker downloads the newer APK ahead on an unmetered network
  (battery and storage not low, three tries); the download writes a .part
  file and keeps it under the build's name only once its size and sha256
  match, so Install is usually one tap.
- Back from allowing installs, the update the listener asked for carries
  on (kept in SavedStateHandle) instead of asking them to tap again.
- An offer needs a higher code AND a different name: the same app built
  again at a later minute is not offered.
- Server: /api/client/version reports the APK's sha256, worked out from
  the served file and cached by size and mtime.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-26 07:46:01 -04:00
bvandeusenandClaude Opus 5.5 de41169ccb fix(android): detekt — no four-way destructuring, single-track loader within the function cap (#6451)
The library-duration fallback rides on the poll outcome instead of being its
own member, and the adopt path reads the target's fields by name.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-26 07:46:01 -04:00
146 changed files with 10765 additions and 1774 deletions
+40 -103
View File
@@ -2,9 +2,9 @@ name: release
# Builds and pushes the minstrel container image to the Gitea registry.
#
# push to dev → :dev (freshly-built dev APK bundled)
# push to main → :latest + :<sha> (latest-release APK bundled)
# push tag vYYYY.MM.DD.HHMM → :latest (fresh APK bundled)
# push to dev → :dev (freshly-built dev APK bundled)
# push to main → :latest + :<sha> (freshly-built stable APK bundled)
# push tag vYYYY.MM.DD.HHMM → :latest (fresh APK bundled, and attached to the Release)
# workflow_dispatch → manual trigger (same rules based on the ref)
#
# That is the whole tag map.
@@ -41,6 +41,12 @@ name: release
# image in the same run from the same commit, and the channel cannot go
# stale against its own artifact.
#
# Every branch build carries the app built from its own commit, so a merge to
# main is all it takes to put a new app in front of stable phones: `:latest`
# serves it from /api/client/apk and the in-app update offers it. A release
# is an optional bookmark — a named point in history with the APK attached
# for download — never a step the app has to wait for.
#
# Release model: the tag IS the artifact's version name with a `v` in front.
# `v2026.09.10.1432` and `2026.09.10.1432` are the same string, derived from
# the tagged commit's UTC timestamp — so there is no mismatch to reconcile
@@ -56,20 +62,14 @@ name: release
# :latest is updated by every main push AND every tag push, so it always
# reflects the newest blessed image.
#
# APK pipeline: on tag pushes the android-release job builds + signs the
# Android APK and uploads it as a workflow artifact. The image-release
# job declares `needs: android-release`, so the docker image cannot
# start building until the APK is guaranteed-ready — no polling, no
# race, no silent-failure mode. Attaching the APK to the gitea Release is
# its own job (release-assets), behind the test gate below.
#
# :latest always carries an APK. Because every main push also moves
# :latest (not just tags), a main build with no APK would silently strip
# the in-app update channel off :latest until the next release. So on
# non-tag builds image-release pulls the MOST RECENT release's signed APK
# AND the version sidecar published beside it — the recorded values, not
# recomputed ones — so no rebuild is needed, just a rebundle. Tag builds
# keep bundling their own freshly-built APK.
# APK pipeline: on every push — dev, main and tags alike — the
# android-release job builds + signs the Android APK and uploads it as a
# workflow artifact. The image-release job declares `needs: android-release`
# and requires its success, so the docker image cannot start building until
# the APK is guaranteed-ready — no polling, no race, no silent-failure mode,
# and no image that moves a channel without an app inside it. Attaching the
# APK to the gitea Release is its own job (release-assets), tags only, behind
# the test gate below.
#
# THE GATE. Every verifying lane lives in this file —
# Go (vet, lint, short tests), the Postgres integration suite, the web app
@@ -400,11 +400,14 @@ jobs:
# ------------------------------------------------------------ artifacts --
android-release:
name: Build signed APK (releases and dev)
# Also builds on `dev`, which is what makes a test channel possible at
# all: without it the only way to get a build onto a phone is to cut a
# release, which quietly turns `main` into the staging area.
if: startsWith(github.ref, 'refs/tags/v') || github.ref == 'refs/heads/dev'
name: Build signed APK
# Builds on every ref, so each channel's image carries the app from its
# own commit: `dev` for the test channel, `main` for stable phones, and a
# tag for the copy attached to its Release.
if: >-
startsWith(github.ref, 'refs/tags/v')
|| github.ref == 'refs/heads/dev'
|| github.ref == 'refs/heads/main'
runs-on: flutter-ci
container:
image: git.fabledsword.com/bvandeusen/ci-android:36
@@ -452,7 +455,7 @@ jobs:
# The derivation lives in ci/version.sh, not here, so it can be
# executed by a test on every push. Anything inline in this file is
# unverifiable until a release is already running.
out="$(ci/version.sh HEAD)"
out="$(ci/version.sh --apk HEAD)"
printf '%s\n' "${out}" >> "$GITHUB_OUTPUT"
# The channel is a property of the LANE, not of the commit, which is
@@ -601,9 +604,9 @@ jobs:
path: release-apk/
- name: Attach APK to gitea Release
# Tag releases only. A dev build has no Release to hang assets on and
# does not need one — the :dev image bundles the APK, and the server
# serves it from /api/client/apk like any other.
# Tag releases only. A branch build has no Release to hang assets on
# and does not need one — its image bundles the APK, and the server
# serves it from /api/client/apk.
shell: bash
env:
CI_TOKEN: ${{ secrets.CI_TOKEN }}
@@ -667,10 +670,9 @@ jobs:
image-release:
name: Build + push container image
# Every lane must have SUCCEEDED, each named here. Then the
# APK: tag and dev pushes build one and it must have succeeded; main
# pushes skip android-release and bundle the latest release's APK
# instead, so for main alone a skipped android-release is expected.
# Every lane must have SUCCEEDED, each named here, and so must the APK
# build: every ref bundles the APK built from its own commit, so an
# image without one would move a channel with no app for phones to take.
needs: [go, integration, web, android, govulncheck, android-release]
if: >-
${{
@@ -680,8 +682,7 @@ jobs:
&& needs.web.result == 'success'
&& needs.android.result == 'success'
&& needs.govulncheck.result == 'success'
&& (needs.android-release.result == 'success'
|| (needs.android-release.result == 'skipped' && github.ref == 'refs/heads/main')) }}
&& needs.android-release.result == 'success' }}
runs-on: go-ci
container:
image: git.fabledsword.com/bvandeusen/ci-go:1.26
@@ -747,9 +748,10 @@ jobs:
# publish the moving channel tag and never a commit-addressable
# one.
#
# :latest must move here rather than waiting for the next main
# push, or the channel would carry the PREVIOUS release's APK
# indefinitely — a channel that cannot refresh itself.
# :latest moves here so the server hands out the same APK build
# that is attached to the Release: a phone that installed from the
# Release page and one that updated in-app hold one build, with one
# ordering key. Cut a release on main's tip, as :latest follows it.
CHANNEL=stable
echo "args=-t ${IMAGE}:latest" >> "$GITHUB_OUTPUT"
echo "::notice::Release build ${VERSION}: refreshing :latest around the new APK"
@@ -782,12 +784,8 @@ jobs:
| docker login git.fabledsword.com -u "${{ github.actor }}" --password-stdin
- name: Download signed APK artifact
# Tag and dev pushes — android-release just produced this. Only `main`
# takes the "Bundle latest release APK" path below, because it is the
# one ref that moves a channel without building an APK of its own.
if: >-
steps.guard.outputs.ready == 'true' &&
(startsWith(github.ref, 'refs/tags/v') || github.ref == 'refs/heads/dev')
# android-release just produced this, on every ref.
if: steps.guard.outputs.ready == 'true'
# Consuming half of the pair: stock download-artifact, which works here for
# the same reason as the upload (gitea/runner 3.x edits the GHES refusal
# out of the bundle). v8 runs on node24, which every CI-runner image
@@ -798,9 +796,7 @@ jobs:
path: client/
- name: Stage bundled APK + version sidecar
if: >-
steps.guard.outputs.ready == 'true' &&
(startsWith(github.ref, 'refs/tags/v') || github.ref == 'refs/heads/dev')
if: steps.guard.outputs.ready == 'true'
shell: bash
env:
# All three pulled from android-release's outputs so the sidecar the
@@ -821,65 +817,6 @@ jobs:
cat client/minstrel.apk.version
ls -lh client/
- name: Bundle latest release APK (non-tag :latest builds)
# Main pushes don't build an APK, but they DO move :latest — so
# without this the in-app update channel would vanish from :latest
# until the next tag. Pull the most-recent release's signed APK and
# the sidecar published beside it, so what the server reports is what
# that build actually recorded rather than something re-derived here.
# Degrades to an empty client/ (404 update channel) — never a wrong
# version — if no release or APK asset can be resolved. That
# degradation only actually works because the greps below carry
# `|| true`; under the runner's default pipefail a non-matching grep
# kills the step instead of falling through to the empty-case branch.
if: steps.guard.outputs.ready == 'true' && github.ref == 'refs/heads/main'
shell: bash
env:
CI_TOKEN: ${{ secrets.CI_TOKEN }}
run: |
set -eu
REPO="${GITHUB_REPOSITORY}"
REL_JSON="$(curl -fsSL -H "Authorization: token ${CI_TOKEN}" \
"https://git.fabledsword.com/api/v1/repos/${REPO}/releases/latest" || true)"
if [ -z "${REL_JSON}" ]; then
echo "::notice::no published release — image ships without bundled APK"; exit 0
fi
# `|| true` on every one of these, and it is load-bearing rather
# than defensive habit. The runner already invokes this shell as
# `bash -e -o pipefail`, so a pipeline whose grep matches NOTHING
# exits non-zero even though `head` succeeded — and the step dies at
# the assignment, before ever reaching the `if` written to handle the
# empty case. Every "degrades gracefully" branch below is unreachable
# without this.
TAG="$(printf '%s' "${REL_JSON}" | grep -oP '"tag_name":\s*"\K[^"]+' | head -1)" || true
APK_URL="$(printf '%s' "${REL_JSON}" | grep -oP '"browser_download_url":\s*"\K[^"]+' | grep -E '\.apk$' | head -1)" || true
if [ -z "${TAG}" ] || [ -z "${APK_URL}" ]; then
echo "::notice::latest release '${TAG:-?}' has no APK asset — image ships without bundled APK"; exit 0
fi
curl -fsSL -H "Authorization: token ${CI_TOKEN}" -o client/minstrel.apk "${APK_URL}"
# Take the version the release RECORDED rather than recomputing it.
# Re-deriving it from the tagged commit would keep the formula in two
# files that must stay in step, and could only ever recover the name —
# the ordering key is build-time minutes and exists nowhere after that
# build ends.
SIDECAR_URL="$(printf '%s' "${REL_JSON}" | grep -oP '"browser_download_url":\s*"\K[^"]+' | grep -E '\.apk\.version$' | head -1)" || true
if [ -n "${SIDECAR_URL}" ]; then
curl -fsSL -H "Authorization: token ${CI_TOKEN}" -o client/minstrel.apk.version "${SIDECAR_URL}"
cat client/minstrel.apk.version
else
# Releases published before sidecars were attached. Their name is
# still recoverable from the tag, but their ordering key genuinely
# is not — so it is reported ABSENT rather than guessed. A wrong
# key is an install the platform refuses; an absent one just tells
# the client to fall back to comparing names, which is exactly
# what those builds already do.
echo "::notice::release ${TAG} predates the version sidecar — bundling with name only, no ordering key"
printf '{"name":"%s","code":null,"channel":"stable"}\n' "${TAG#v}" > client/minstrel.apk.version
fi
echo "::notice::bundled release APK from ${TAG}"
ls -lh client/
- name: Build and push
if: steps.guard.outputs.ready == 'true'
# --pull: the Dockerfile's base images are floating tags (golang:1.26,
@@ -24,6 +24,7 @@ import androidx.core.content.ContextCompat
import androidx.hilt.navigation.compose.hiltViewModel
import androidx.lifecycle.compose.collectAsStateWithLifecycle
import androidx.lifecycle.lifecycleScope
import androidx.navigation.NavHostController
import androidx.navigation.compose.rememberNavController
import com.fabledsword.minstrel.auth.AuthStore
import com.fabledsword.minstrel.auth.ui.AuthGateViewModel
@@ -33,6 +34,7 @@ import com.fabledsword.minstrel.connectivity.ServerHealth
import com.fabledsword.minstrel.connectivity.NetworkStatusController
import com.fabledsword.minstrel.nav.DetailSeedCache
import com.fabledsword.minstrel.nav.LocalDetailSeedCache
import com.fabledsword.minstrel.nav.Login
import com.fabledsword.minstrel.nav.MinstrelNavGraph
import com.fabledsword.minstrel.nav.Notifications
import com.fabledsword.minstrel.nav.NowPlaying
@@ -177,6 +179,7 @@ private fun App(
navController.navigate(route) { launchSingleTop = true }
onOpenedRoute()
}
SignInWhenSessionRejected(gate, navController)
MinstrelNavGraph(
navController = navController,
startDestination = resolved,
@@ -187,6 +190,24 @@ private fun App(
}
}
/**
* When the server ends the session while the app is in use, opens the
* sign-in screen rather than leaving the user on screens whose every
* request fails.
*/
@Composable
private fun SignInWhenSessionRejected(gate: AuthGateViewModel, navController: NavHostController) {
val sessionRejected by gate.sessionRejected.collectAsStateWithLifecycle()
LaunchedEffect(sessionRejected, navController) {
if (!sessionRejected) return@LaunchedEffect
navController.navigate(Login) {
popUpTo(0) { inclusive = true }
launchSingleTop = true
}
gate.consumeSessionRejected()
}
}
@Composable
private fun BootSplash() {
Surface(
@@ -24,7 +24,6 @@ import com.fabledsword.minstrel.player.LikedAudioPinner
import com.fabledsword.minstrel.player.PlayEventsReporter
import com.fabledsword.minstrel.player.PlaybackErrorReporter
import com.fabledsword.minstrel.player.ResumeController
import com.fabledsword.minstrel.update.data.UpdateBannerController
import com.fabledsword.minstrel.connectivity.NetworkStatusController
import dagger.hilt.android.HiltAndroidApp
import kotlinx.coroutines.CoroutineScope
@@ -155,14 +154,6 @@ class MinstrelApplication :
*/
@Suppress("unused") @Inject lateinit var networkStatusController: NetworkStatusController
/**
* Same construct-the-singleton trick — UpdateBannerController polls
* /api/client/version at launch + every 24h and drives the shell's
* soft "update available" banner. Without this @Inject the poll
* loop would never start and the banner would never surface.
*/
@Suppress("unused") @Inject lateinit var updateBannerController: UpdateBannerController
/**
* Same construct-the-singleton trick — FreshnessSweeper's init
* block starts a 30-min sweep loop that re-fetches cached
@@ -8,9 +8,9 @@ import javax.inject.Singleton
/**
* Attaches the session cookie from [AuthStore] to Minstrel-server
* requests, captures Set-Cookie from successful responses (login
* flow), and clears the store on 401 so downstream code can react
* to logout.
* requests (waiting for the stored one to load at launch), captures
* Set-Cookie from successful responses (login flow), and signs out when
* the server refuses the cookie a request carried.
*
* Scoped to the [BaseUrlInterceptor.PLACEHOLDER_HOST] sentinel host
* the same way BaseUrlInterceptor is. The shared OkHttpClient also
@@ -32,7 +32,7 @@ class AuthCookieInterceptor @Inject constructor(
// session. Pass through untouched.
return chain.proceed(original)
}
val cookie = authStore.sessionCookie.value
val cookie = authStore.sessionCookieForRequest()
val request = original.newBuilder().apply {
if (!cookie.isNullOrEmpty()) header("Cookie", cookie)
}.build()
@@ -40,7 +40,11 @@ class AuthCookieInterceptor @Inject constructor(
val response = chain.proceed(request)
if (response.code == HTTP_UNAUTHORIZED) {
authStore.setSessionCookie(null)
// Only a refusal of the cookie this request carried ends the
// session. A request sent without one says nothing about the
// stored login, and a refusal of an older cookie must not
// discard one a sign-in has stored since.
if (!cookie.isNullOrEmpty()) authStore.dropRejectedSessionCookie(cookie)
} else if (response.isSuccessful) {
// Capture the first Set-Cookie pair on login; subsequent
// responses that include Set-Cookie keep the store fresh.
@@ -89,8 +89,11 @@ class AuthController @Inject constructor(
* + persisted userJson. Carries account-level flags (debug mode)
* that aren't in the login response. Best-effort: a network failure
* leaves the cached identity untouched. No-op when signed out.
* Waits for the stored login to load first: at launch this runs before
* it has, and would otherwise read the user as signed out and skip.
*/
suspend fun refreshProfile() {
authStore.awaitSessionHydrated()
if (!isSignedIn) return
val p = runCatching { meApi.getProfile() }.getOrNull() ?: return
val user = UserRef(
@@ -12,6 +12,7 @@ import kotlinx.coroutines.flow.MutableStateFlow
import kotlinx.coroutines.flow.StateFlow
import kotlinx.coroutines.flow.asStateFlow
import kotlinx.coroutines.launch
import kotlinx.coroutines.runBlocking
import kotlinx.coroutines.sync.Mutex
import kotlinx.coroutines.sync.withLock
import kotlinx.coroutines.withTimeoutOrNull
@@ -148,6 +149,49 @@ class AuthStore @Inject constructor(
scope.launch { cookieLock.withLock { storeCookie(value) } }
}
/**
* The cookie to attach to an outgoing request. Blocks the calling
* (OkHttp) thread until the stored cookie has loaded, up to
* [HYDRATION_DEADLINE_MS], so a request made in the first moment after
* launch carries the login instead of going out without it. Once
* hydration has finished this is a plain read.
*/
fun sessionCookieForRequest(): String? {
if (!cookieHydration.isCompleted) {
runBlocking { withTimeoutOrNull(HYDRATION_DEADLINE_MS) { cookieHydration.await() } }
}
return sessionCookieState.value
}
/**
* True once the server has refused the current session (see
* [dropRejectedSessionCookie]) and the UI has not yet sent the user to
* sign in. Held rather than fired, so a refusal that lands while the app
* is in the background is still acted on when it comes back. A sign-out
* the user asked for does not set it.
*/
private val sessionRejectedState = MutableStateFlow(false)
val sessionRejected: StateFlow<Boolean> = sessionRejectedState.asStateFlow()
/** The UI has opened the sign-in screen for a refused session. */
fun consumeSessionRejected() {
sessionRejectedState.value = false
}
/**
* Signs out because the server answered 401 to a request that carried
* [rejected]. Does nothing when the store has since moved on to another
* cookie (a sign-in landed meanwhile), so a refusal of an old session
* never discards a new one. What is persisted is read inside the lock,
* so a sign-in racing this call is the value that ends up stored.
*/
fun dropRejectedSessionCookie(rejected: String) {
if (!sessionCookieState.compareAndSet(rejected, null)) return
cookieTouched = true
scope.launch { cookieLock.withLock { storeCookie(sessionCookieState.value) } }
sessionRejectedState.value = true
}
private suspend fun hydrateSessionCookie() = cookieLock.withLock {
val legacy = runCatching { dao.get()?.sessionCookie }.getOrNull()
if (cookieTouched) return@withLock
@@ -38,6 +38,15 @@ class AuthGateViewModel @Inject constructor(
private val internal = MutableStateFlow<Any?>(null)
val startDestination: StateFlow<Any?> = internal.asStateFlow()
/**
* True when the server has ended the session while the app was in use;
* the root NavHost answers by opening the sign-in screen, then calls
* [consumeSessionRejected].
*/
val sessionRejected: StateFlow<Boolean> = authStore.sessionRejected
fun consumeSessionRejected() = authStore.consumeSessionRejected()
init {
viewModelScope.launch {
authStore.awaitSessionHydrated()
@@ -70,6 +70,9 @@ class DiagnosticsUploader @Inject constructor(
/** Coalesces concurrent triggers via tryLock — a drain in flight wins. */
private suspend fun drainSafe() {
// A trigger at launch can fire before the stored login has loaded;
// wait for it, so a signed-in user is not read as signed out.
authStore.awaitSessionHydrated()
val clientId = authStore.clientId.value
// Signed out / no client id yet → nothing to do. Single guard keeps
// the return count within the detekt gate.
@@ -102,6 +102,8 @@ import com.fabledsword.minstrel.shared.widgets.PullToRefreshScaffold
import com.fabledsword.minstrel.shared.widgets.SkeletonAlbumTile
import com.fabledsword.minstrel.shared.widgets.SkeletonArtistTile
import com.fabledsword.minstrel.shared.widgets.SkeletonSectionHeader
import com.fabledsword.minstrel.update.ui.UpdateNotices
import com.fabledsword.minstrel.update.ui.UpdateViewModel
import dagger.hilt.android.lifecycle.HiltViewModel
import kotlinx.coroutines.async
import kotlinx.coroutines.channels.Channel
@@ -553,6 +555,7 @@ class HomeViewModel @Inject constructor(
fun HomeScreen(
navController: NavHostController,
viewModel: HomeViewModel = hiltViewModel(),
updates: UpdateViewModel = hiltViewModel(),
) {
val snackbarHostState = remember { SnackbarHostState() }
LaunchedEffect(Unit) {
@@ -575,7 +578,11 @@ fun HomeScreen(
val offline by viewModel.offline.collectAsStateWithLifecycle()
val updating by viewModel.isUpdating.collectAsStateWithLifecycle()
PullToRefreshScaffold(
onRefresh = { viewModel.refreshFromPull() },
onRefresh = {
// A pull asks about a newer app too, so one shows up at once.
updates.check()
viewModel.refreshFromPull()
},
modifier = Modifier.fillMaxSize().padding(inner),
) {
Box(Modifier.fillMaxSize()) {
@@ -590,6 +597,7 @@ fun HomeScreen(
// hide behind an "Updating your mixes…" wipe that stays up
// until the screen has actually stopped moving.
UpdatingVeil(visible = updating)
UpdateNotices(updates)
}
}
}
@@ -773,6 +781,13 @@ private fun HomeSuccessContent(
onPlayPool = onPlayPool,
onPlayPlaylist = onPlayPlaylist,
)
genreMixSection(
playlists = sections.playlists,
offline = offline,
onPlaylistClick = onPlaylistClick,
onPlayPool = onPlayPool,
onPlayPlaylist = onPlayPlaylist,
)
youMightLikeSection(
albums = sections.youMightLikeAlbums,
artists = sections.youMightLikeArtists,
@@ -825,6 +840,31 @@ private fun LazyListScope.songsLikeSection(
}
}
/**
* The "Your genres" row: one mix per genre the user listens to. Hidden
* when there is nothing to show, built or cached.
*/
private fun LazyListScope.genreMixSection(
playlists: List<PlaylistRef>,
offline: Boolean,
onPlaylistClick: (String) -> Unit,
onPlayPool: (OfflinePoolKind) -> Unit,
onPlayPlaylist: suspend (PlaylistRef) -> Unit,
) {
val rowItems = buildGenreMixRow(playlists, offline)
if (rowItems.isEmpty()) return
item {
PlaylistsRow(
title = "Your genres",
rowItems = rowItems,
offline = offline,
onPlaylistClick = onPlaylistClick,
onPlayPool = onPlayPool,
onPlayPlaylist = onPlayPlaylist,
)
}
}
private fun LazyListScope.recentlyAddedSection(
albums: List<HomeTile<AlbumRef>>,
onAlbumClick: (String) -> Unit,
@@ -1225,6 +1265,25 @@ internal fun buildSongsLikeRow(
}
}
/**
* Builds the "Your genres" row: every genre mix, in name order so the
* shelf reads like a stable channel list. No placeholders — a genre mix
* needs listening history or a genre-tagged library, so none built means
* nothing to show. Offline, fully-cached mixes come first and the rest are
* greyed. Mirrors the web Home's genreMixPlaylists.
*/
internal fun buildGenreMixRow(
owned: List<PlaylistRef>,
offline: Boolean,
): List<PlaylistRowItem> {
val mixes = owned
.filter { it.systemVariant == "genre_mix" }
.sortedBy { it.name.lowercase() }
if (!offline) return mixes.map { PlaylistRowItem.Real(it) }
val (available, greyed) = mixes.partition { !it.unavailableOffline }
return (available + greyed).map { PlaylistRowItem.Real(it) }
}
/**
* Curated real-playlist order (system primaries, then secondary, then user).
* Must mirror [buildOnlineRow]'s slot order — the offline row reuses this and
@@ -26,10 +26,11 @@ data class PlaylistRef(
/**
* Whether the "Regenerate" affordance applies: every system playlist
* EXCEPT songs_like_artist (the server's songs-like family exposes no
* refresh trigger; rebuilds are scheduler-driven).
* EXCEPT the kinds with several playlists per user (songs_like_artist,
* genre_mix), which the server can't address by kind; those rebuild on
* the daily schedule and play by playlist id.
*/
val refreshable: Boolean get() = isSystem && systemVariant != "songs_like_artist"
val refreshable: Boolean get() = isSystem && systemVariant !in PER_PLAYLIST_SYSTEM_VARIANTS
/**
* Can't be relied on while offline — either it needs the live server to
@@ -39,6 +40,9 @@ data class PlaylistRef(
val unavailableOffline: Boolean get() = refreshable || !fullyCached
}
/** System kinds with several playlists per user, played by playlist id. */
private val PER_PLAYLIST_SYSTEM_VARIANTS = setOf("songs_like_artist", "genre_mix")
/**
* One row of a playlist's tracks.
*
@@ -15,7 +15,9 @@ package com.fabledsword.minstrel.models
* verbatim rather than validated, so an unexpected value is shown rather
* than dropped.
*
* [apkUrl] is server-relative (e.g. `/api/client/apk`).
* [apkUrl] is server-relative (e.g. `/api/client/apk`). [sha256] is the
* served file's digest, in hex; empty from a server that predates it, and
* then a download is judged whole by its size alone.
*/
data class UpdateInfo(
val version: String,
@@ -23,4 +25,5 @@ data class UpdateInfo(
val channel: String?,
val apkUrl: String,
val sizeBytes: Long,
val sha256: String,
)
@@ -26,4 +26,5 @@ data class UpdateInfoWire(
val channel: String? = null,
@SerialName("apk_url") val apkUrl: String = "/api/client/apk",
@SerialName("size_bytes") val sizeBytes: Long = 0,
val sha256: String = "",
)
@@ -9,23 +9,17 @@ import android.os.Build
import android.provider.Settings
import androidx.activity.compose.rememberLauncherForActivityResult
import androidx.activity.result.contract.ActivityResultContracts
import androidx.compose.foundation.clickable
import androidx.compose.foundation.layout.Arrangement
import androidx.compose.foundation.layout.Box
import androidx.compose.foundation.layout.Column
import androidx.compose.foundation.layout.Row
import androidx.compose.foundation.layout.fillMaxSize
import androidx.compose.foundation.layout.fillMaxWidth
import androidx.compose.foundation.layout.padding
import androidx.compose.foundation.layout.width
import androidx.compose.foundation.rememberScrollState
import androidx.compose.foundation.selection.toggleable
import androidx.compose.foundation.verticalScroll
import androidx.compose.material3.Checkbox
import androidx.compose.material3.Icon
import androidx.compose.material3.MaterialTheme
import androidx.compose.material3.Scaffold
import androidx.compose.material3.Switch
import androidx.compose.material3.Text
import androidx.compose.runtime.Composable
import androidx.compose.runtime.getValue
@@ -35,7 +29,6 @@ import androidx.compose.runtime.setValue
import androidx.compose.ui.Alignment
import androidx.compose.ui.Modifier
import androidx.compose.ui.platform.LocalContext
import androidx.compose.ui.semantics.Role
import androidx.compose.ui.semantics.contentDescription
import androidx.compose.ui.semantics.semantics
import androidx.compose.ui.text.style.TextAlign
@@ -45,8 +38,6 @@ import androidx.hilt.navigation.compose.hiltViewModel
import androidx.lifecycle.compose.LifecycleResumeEffect
import androidx.lifecycle.compose.collectAsStateWithLifecycle
import androidx.navigation.NavHostController
import com.composables.icons.lucide.ChevronRight
import com.composables.icons.lucide.Lucide
import com.fabledsword.minstrel.api.endpoints.NotificationKindSettingWire
import com.fabledsword.minstrel.api.endpoints.NotificationSettingsWire
import com.fabledsword.minstrel.nav.NotificationSettings
@@ -54,13 +45,19 @@ import com.fabledsword.minstrel.notifications.data.NotificationChannel
import com.fabledsword.minstrel.shared.widgets.LoadingCentered
import com.fabledsword.minstrel.shared.widgets.MinstrelTopAppBar
import com.fabledsword.minstrel.shared.widgets.ShellContentWindowInsets
import com.fabledsword.minstrel.theme.Muted
import com.fabledsword.minstrel.theme.RowChevron
import com.fabledsword.minstrel.theme.Section
import com.fabledsword.minstrel.theme.SectionEdge
import com.fabledsword.minstrel.theme.SectionGap
import com.fabledsword.minstrel.theme.SettingRow
import com.fabledsword.minstrel.theme.SwitchRow
private val CELL_WIDTH = 56.dp
private val CELL_WIDTH = 48.dp
/**
* Per-user notification settings: a row per kind with a box for each
* channel. Rows read as a menu; anything that needs explaining
* is one short line.
* Per-user notification settings: what this device does, then a row per
* kind with a box for each channel.
*/
@Composable
fun NotificationSettingsScreen(
@@ -109,38 +106,36 @@ private fun SettingsBody(
modifier = modifier
.fillMaxSize()
.verticalScroll(rememberScrollState())
.padding(vertical = 8.dp),
.padding(horizontal = SectionEdge, vertical = SectionEdge),
verticalArrangement = Arrangement.spacedBy(SectionGap),
) {
deviceRows()
HeaderRow()
settings.kinds.filter { !it.adminOnly }.forEach { KindRow(it, settings.emailAvailable, onSet) }
Section("This device") { deviceRows() }
Section("Requests", trailing = { ChannelHeaders() }) {
settings.kinds.filter { !it.adminOnly }.forEach { KindRow(it, settings.emailAvailable, onSet) }
if (!settings.emailAvailable) {
Muted(emailUnavailableLine(settings.emailUnavailableReason, isAdmin))
}
}
val adminKinds = settings.kinds.filter { it.adminOnly }
if (adminKinds.isNotEmpty()) {
SectionLabel("Library health")
adminKinds.forEach { KindRow(it, settings.emailAvailable, onSet) }
}
if (!settings.emailAvailable) {
Muted(emailUnavailableLine(settings.emailUnavailableReason, isAdmin))
Section("Library health", trailing = { ChannelHeaders() }) {
adminKinds.forEach { KindRow(it, settings.emailAvailable, onSet) }
}
}
}
}
/** The channel names over the boxes, in the section's title line. */
@Composable
private fun HeaderRow() {
Row(
modifier = Modifier.fillMaxWidth().padding(horizontal = 16.dp, vertical = 4.dp),
verticalAlignment = Alignment.CenterVertically,
) {
Box(Modifier.weight(1f))
NotificationChannel.entries.forEach { c ->
Text(
text = channelLabel(c),
modifier = Modifier.width(CELL_WIDTH),
style = MaterialTheme.typography.labelMedium,
color = MaterialTheme.colorScheme.onSurfaceVariant,
textAlign = TextAlign.Center,
)
}
private fun ChannelHeaders() {
NotificationChannel.entries.forEach { c ->
Text(
text = channelLabel(c),
modifier = Modifier.width(CELL_WIDTH),
style = MaterialTheme.typography.labelMedium,
color = MaterialTheme.colorScheme.onSurfaceVariant,
textAlign = TextAlign.Center,
)
}
}
@@ -151,11 +146,7 @@ private fun KindRow(
onSet: (String, NotificationChannel, Boolean) -> Unit,
) {
val label = kindLabel(row.kind)
Row(
modifier = Modifier.fillMaxWidth().padding(horizontal = 16.dp),
verticalAlignment = Alignment.CenterVertically,
) {
Text(label, modifier = Modifier.weight(1f), style = MaterialTheme.typography.bodyLarge)
SettingRow(title = label) {
NotificationChannel.entries.forEach { c ->
val enabled = channelEnabled(row, c, emailAvailable)
Box(Modifier.width(CELL_WIDTH), contentAlignment = Alignment.Center) {
@@ -204,60 +195,24 @@ private fun DeviceRows(backgroundDelivery: Boolean, onBackgroundDelivery: (Boole
@Composable
private fun BackgroundDeliveryRow(checked: Boolean, onChange: (Boolean) -> Unit) {
Row(
modifier = Modifier
.fillMaxWidth()
.toggleable(value = checked, role = Role.Switch, onValueChange = onChange)
.padding(horizontal = 16.dp, vertical = 12.dp),
verticalAlignment = Alignment.CenterVertically,
horizontalArrangement = Arrangement.spacedBy(8.dp),
) {
Column(Modifier.weight(1f)) {
Text("Notifications when the app is closed", style = MaterialTheme.typography.bodyLarge)
// The ongoing notice is the price of not using Google's push;
// said plainly rather than left to be discovered.
Muted("Keeps a quiet notice in the shade, in place of Google's push", padded = false)
}
Switch(checked = checked, onCheckedChange = null)
}
// The ongoing notice is the price of not using Google's push; said
// plainly rather than left to be discovered.
SwitchRow(
title = "Notifications when the app is closed",
summary = "Keeps a quiet notice in the shade, in place of Google's push",
checked = checked,
onCheckedChange = onChange,
)
}
@Composable
private fun PhoneBlockedRow(onClick: () -> Unit) {
Row(
modifier = Modifier
.fillMaxWidth()
.clickable(onClick = onClick)
.padding(horizontal = 16.dp, vertical = 12.dp),
verticalAlignment = Alignment.CenterVertically,
horizontalArrangement = Arrangement.spacedBy(8.dp),
) {
Column(Modifier.weight(1f)) {
Text("Phone alerts are off", style = MaterialTheme.typography.bodyLarge)
Muted("Allow notifications for Minstrel in system settings", padded = false)
}
Icon(Lucide.ChevronRight, contentDescription = null, tint = MaterialTheme.colorScheme.onSurfaceVariant)
}
}
@Composable
private fun SectionLabel(text: String) {
Text(
text = text,
modifier = Modifier.padding(start = 16.dp, end = 16.dp, top = 16.dp, bottom = 4.dp),
style = MaterialTheme.typography.titleSmall,
color = MaterialTheme.colorScheme.onSurfaceVariant,
)
}
@Composable
private fun Muted(text: String, padded: Boolean = true) {
Text(
text = text,
modifier = if (padded) Modifier.padding(horizontal = 16.dp, vertical = 12.dp) else Modifier,
style = MaterialTheme.typography.bodySmall,
color = MaterialTheme.colorScheme.onSurfaceVariant,
)
SettingRow(
title = "Phone alerts are off",
summary = "Allow notifications for Minstrel in system settings",
tone = MaterialTheme.colorScheme.error,
onClick = onClick,
) { RowChevron() }
}
private fun openAppNotificationSettings(context: Context) {
@@ -216,24 +216,56 @@ class MinstrelForwardingPlayer(
// drops via isLoadingUpnp() = true; the OutputPickerController
// observes the uiState.queue change and runs the resync (re-clears
// Sonos's native queue + AddURIToQueue the new tracks + Play).
//
// The SAME tracks set again (play on the album already queued, from
// another track) is not a new queue: the renderer already holds it, and
// the uiState collector -- which keys on the track ids -- would never see
// a change to resync on, leaving the cast stuck "loading". It is a move
// within the queue, so the renderer is pointed at the new track and the
// play() that follows goes through.
override fun setMediaItems(mediaItems: List<MediaItem>) {
val sameQueue = holdsSameItems(mediaItems)
super.setMediaItems(mediaItems)
markPendingResyncIfRemote()
afterQueueSet(sameQueue)
}
override fun setMediaItems(mediaItems: List<MediaItem>, resetPosition: Boolean) {
val sameQueue = holdsSameItems(mediaItems)
super.setMediaItems(mediaItems, resetPosition)
markPendingResyncIfRemote()
afterQueueSet(sameQueue)
}
override fun setMediaItems(mediaItems: List<MediaItem>, startIndex: Int, startPositionMs: Long) {
val sameQueue = holdsSameItems(mediaItems)
super.setMediaItems(mediaItems, startIndex, startPositionMs)
markPendingResyncIfRemote()
afterQueueSet(sameQueue)
}
private fun markPendingResyncIfRemote() {
val wasActive = holder.active.value ?: return
private fun holdsSameItems(mediaItems: List<MediaItem>): Boolean =
mediaItems.size == delegate.mediaItemCount &&
mediaItems.indices.all { mediaItems[it].mediaId == delegate.getMediaItemAt(it).mediaId }
private fun afterQueueSet(sameQueue: Boolean) {
val active = holder.active.value ?: return
if (!sameQueue) {
markPendingResync(active)
return
}
val index = delegate.currentMediaItemIndex
val positionMs = delegate.currentPosition.coerceAtLeast(0L)
Timber.w(
"setMediaItems with the queue %s already holds -- moving to track %d",
active.routeName,
index + 1,
)
when (active.queueMode) {
QueueMode.SINGLE_TRACK -> startSingleTrack(active, index, positionMs)
QueueMode.NATIVE_QUEUE -> seekNativeQueue(active, index, positionMs)
}
}
private fun markPendingResync(wasActive: ActiveUpnp) {
Timber.w(
"setMediaItems while UPnP active (%s) -- marking pending resync",
wasActive.routeName,
@@ -332,11 +364,15 @@ class MinstrelForwardingPlayer(
super.seekTo(mediaItemIndex, positionMs)
when (active.queueMode) {
QueueMode.SINGLE_TRACK -> playSingleTrackAt(active, mediaItemIndex, positionMs)
QueueMode.NATIVE_QUEUE -> sendNativeQueueCommand(active) {
active.avTransport.seekToTrack(mediaItemIndex + 1)
if (positionMs > 0L) {
active.avTransport.seek(positionMs)
}
QueueMode.NATIVE_QUEUE -> seekNativeQueue(active, mediaItemIndex, positionMs)
}
}
private fun seekNativeQueue(active: ActiveUpnp, index: Int, positionMs: Long) {
sendNativeQueueCommand(active) {
active.avTransport.seekToTrack(index + 1)
if (positionMs > 0L) {
active.avTransport.seek(positionMs)
}
}
}
@@ -747,7 +783,7 @@ class MinstrelForwardingPlayer(
* the library's duration, so the scrubber has an end to run to.
*/
private fun durationFor(info: PositionInfo, single: SingleTrackLoader.PollOutcome?): Long =
if (single != null && info.trackDurationMs <= 0L) singleTrack.knownDurationMs() else info.trackDurationMs
if (single != null && info.trackDurationMs <= 0L) single.knownDurationMs else info.trackDurationMs
/**
* Ask the watchdog what to make of this poll, and act on its answer.
@@ -383,23 +383,27 @@ class OutputPickerController @Inject constructor(
resyncSingleTrack(upnpRoute, transport, newQueue, newCurrentIndex)
return@withLock
}
val handledIncrementally = runCatching {
// The incremental diff is for a queue changed under a cast that kept
// running (play next, an append, a removal): it leaves the playing
// track alone. A queue REPLACED through setMediaItems has already
// cleared active, and the listener asked for its start track and for
// it to play -- both of which the diff would ignore, and with active
// never restored the app would think it had gone back to the phone
// while the speaker played on. That case reloads in full below.
val stillActive = activeUpnpHolder.active.value?.routeId == routeId
val handledIncrementally = stillActive && runCatching {
sonosQueue.tryIncrementalResync(transport, oldIds, newQueue)
}.getOrElse { e ->
Timber.w(e, "Sonos incremental resync errored; falling back to full reload")
false
}
if (handledIncrementally) {
// Active was never cleared on the incremental path; clear any
// target that markPendingResyncIfRemote set (it didn't, for
// incremental cases that don't go through setMediaItems, but
// belt-and-suspenders).
activeUpnpHolder.setTarget(null)
return@withLock
}
// Full rebuild: ensure active is cleared so transport calls drop
// (markPendingResyncIfRemote may already have done this on the
// setMediaItems path).
// (the forwarding player has already done this on the setMediaItems
// path).
if (activeUpnpHolder.active.value != null) {
activeUpnpHolder.set(null)
activeUpnpHolder.setTarget(routeId)
@@ -679,11 +683,11 @@ class OutputPickerController @Inject constructor(
localQueue: List<TrackRef>,
localIndex: Int,
): Boolean {
val (transport, rendering, route, queueMode) = target
val route = target.route
val plan = probeAdoptable(target, localQueue, localIndex) ?: return false
if (queueMode == QueueMode.SINGLE_TRACK) {
if (target.queueMode == QueueMode.SINGLE_TRACK) {
// The renderer holds only this track; arm the one after it.
singleTrack.adopt(transport, localQueue, plan.localIndex, plan.trackUri)
singleTrack.adopt(target.transport, localQueue, plan.localIndex, plan.trackUri)
}
Timber.w(
"UPnP adopt: attaching to live session on %s at track %d (%dms)",
@@ -704,9 +708,9 @@ class OutputPickerController @Inject constructor(
ActiveUpnp(
routeId = route.id,
routeName = route.name,
avTransport = transport,
rendering = rendering,
queueMode = queueMode,
avTransport = target.transport,
rendering = target.rendering,
queueMode = target.queueMode,
),
)
return true
@@ -70,8 +70,10 @@ class SingleTrackLoader @Inject constructor(
* renderer is playing (0 when nothing is loaded). [advanced] is true when
* this poll started the next track itself, so the reading it was given is
* already stale and the caller should not judge a stall from it.
* [knownDurationMs] is the library's duration for that track, for a
* renderer that reports none.
*/
data class PollOutcome(val trackNumber: Int, val advanced: Boolean)
data class PollOutcome(val trackNumber: Int, val advanced: Boolean, val knownDurationMs: Long = 0L)
/** Take [queue] as the one to play, and start [index] on the renderer. */
suspend fun load(transport: AVTransportClient, queue: List<TrackRef>, index: Int) =
@@ -137,9 +139,8 @@ class SingleTrackLoader @Inject constructor(
if (currentIndex !in queue.indices) return@withLock PollOutcome(0, advanced = false)
val moved = followRenderer(trackUri)
armNext(transport)
if (moved) return@withLock PollOutcome(currentIndex + 1, advanced = false)
val known = queue[currentIndex].durationSec * MILLIS_PER_SECOND
endedOnLastPoll = endDetector.onPoll(state, positionMs, durationMs, known)
if (moved) return@withLock PollOutcome(currentIndex + 1, advanced = false, knownDurationMs(queue, currentIndex))
endedOnLastPoll = endDetector.onPoll(state, positionMs, durationMs, knownDurationMs(queue, currentIndex))
val next = currentIndex + 1
val advance = endedOnLastPoll && playIntent && next in queue.indices
if (advance) {
@@ -147,7 +148,7 @@ class SingleTrackLoader @Inject constructor(
runCatching { start(transport, next) }
.onFailure { Timber.w(it, "UPnP single-track: could not start track %d", next + 1) }
}
PollOutcome(currentIndex + 1, advanced = advance)
PollOutcome(currentIndex + 1, advanced = advance, knownDurationMs(queue, currentIndex))
}
/**
@@ -162,10 +163,6 @@ class SingleTrackLoader @Inject constructor(
RemoteStallWatchdog.QueueState.UNKNOWN
}
/** The library's duration for the current track, for a renderer that reports none. */
fun knownDurationMs(): Long =
queue.getOrNull(currentIndex)?.durationSec?.times(MILLIS_PER_SECOND) ?: 0L
private suspend fun start(transport: AVTransportClient, index: Int): Boolean {
if (index !in queue.indices) return false
val token = mint(index)
@@ -240,11 +237,14 @@ class SingleTrackLoader @Inject constructor(
)
private companion object {
const val MILLIS_PER_SECOND = 1_000L
const val ARM_RETRY_POLLS = 10
}
}
/** The library's duration of [queue]'s track at [index], or 0 when there is none. */
private fun knownDurationMs(queue: List<TrackRef>, index: Int): Long =
queue.getOrNull(index)?.durationSec?.times(MILLIS_PER_SECOND) ?: 0L
/**
* Index in [queue] of the track with [id], taking the occurrence nearest
* [preferNear] so a track queued twice does not send the cursor across the
@@ -263,3 +263,4 @@ internal fun streamUriTrackId(uri: String): String? {
}
private const val TRACKS_PATH_MARKER = "/api/tracks/"
private const val MILLIS_PER_SECOND = 1_000L
@@ -171,6 +171,7 @@ private fun systemLabelFor(variant: String?): String = when (variant) {
"new_for_you" -> "New for you"
"on_this_day" -> "On this day"
"first_listens" -> "First listens"
"genre_mix" -> "Genre mix"
null -> ""
else -> variant.replace('_', ' ').replaceFirstChar { it.uppercase() }
}
@@ -1,137 +0,0 @@
package com.fabledsword.minstrel.settings.ui
import androidx.lifecycle.ViewModel
import androidx.lifecycle.viewModelScope
import com.fabledsword.minstrel.BuildConfig
import com.fabledsword.minstrel.api.ErrorCopy
import com.fabledsword.minstrel.models.UpdateInfo
import com.fabledsword.minstrel.update.data.ApkInstaller
import com.fabledsword.minstrel.update.data.InstallStage
import com.fabledsword.minstrel.update.data.UpdateRepository
import com.fabledsword.minstrel.update.data.isBusy
import com.fabledsword.minstrel.update.data.isUpdateAvailable
import com.fabledsword.minstrel.update.data.message
import com.fabledsword.minstrel.update.data.stage
import dagger.hilt.android.lifecycle.HiltViewModel
import kotlinx.coroutines.flow.MutableStateFlow
import kotlinx.coroutines.flow.StateFlow
import kotlinx.coroutines.flow.asStateFlow
import kotlinx.coroutines.flow.update
import kotlinx.coroutines.launch
import java.io.File
import javax.inject.Inject
/**
* One of three terminal states the Check-for-updates button surfaces.
* `Idle` is the pre-check state; `Latest` means the installed build
* matches or exceeds the server's bundled APK; `UpdateAvailable`
* surfaces an "Install vX.Y.Z" button that downloads the APK and
* installs it via [ApkInstaller].
*/
sealed interface UpdateCheckResult {
data object Idle : UpdateCheckResult
data object Latest : UpdateCheckResult
data class UpdateAvailable(val info: UpdateInfo) : UpdateCheckResult
data class Error(val message: String) : UpdateCheckResult
}
data class AboutUiState(
val installedVersion: String = BuildConfig.VERSION_NAME,
// The value the platform installs by, and therefore the one the update
// check must decide on. Held in state rather than read inline so a test
// can drive the comparison without a BuildConfig.
val installedCode: Long = BuildConfig.VERSION_CODE.toLong(),
// A debug build is signed with this machine's debug key, so the server's
// release-signed APK can never install over it. It updates from Android
// Studio instead.
val selfUpdates: Boolean = !BuildConfig.DEBUG,
val isChecking: Boolean = false,
val installStage: InstallStage = InstallStage.IDLE,
val installMessage: String? = null,
val result: UpdateCheckResult = UpdateCheckResult.Idle,
)
/**
* Backs the About card's update controls. "Check for updates" calls
* [UpdateRepository.getLatest], compares versus this build via
* [isUpdateAvailable] — on the ordering key where the server reports one,
* on the name otherwise — and reports the terminal state.
* When an update is available, [install] downloads the APK via
* [ApkInstaller] and installs it — routing the user to the "install
* unknown apps" settings page first when that permission hasn't been
* granted.
*/
@HiltViewModel
class AboutCardViewModel @Inject constructor(
private val repository: UpdateRepository,
private val installer: ApkInstaller,
) : ViewModel() {
private val internal = MutableStateFlow(AboutUiState())
val state: StateFlow<AboutUiState> = internal.asStateFlow()
fun checkForUpdates() {
if (internal.value.isChecking || !internal.value.selfUpdates) return
viewModelScope.launch {
internal.update { it.copy(isChecking = true, installMessage = null) }
val installed = internal.value.installedVersion
val installedCode = internal.value.installedCode
val result = runCatching { repository.getLatest() }
.map { latest ->
if (
isUpdateAvailable(
serverCode = latest.code,
serverName = latest.version,
installedCode = installedCode,
installedName = installed,
)
) {
UpdateCheckResult.UpdateAvailable(latest)
} else {
UpdateCheckResult.Latest
}
}
.getOrElse { e -> UpdateCheckResult.Error(ErrorCopy.fromThrowable(e)) }
internal.update { it.copy(isChecking = false, result = result) }
}
}
fun install(info: UpdateInfo) {
if (internal.value.installStage.isBusy()) return
if (!installer.canInstall()) {
installer.requestInstallPermission()
internal.update {
it.copy(installMessage = "Allow installs for Minstrel, then tap Install again.")
}
return
}
viewModelScope.launch {
internal.update {
it.copy(installStage = InstallStage.DOWNLOADING, installMessage = null)
}
val apk = download(info.apkUrl)
if (apk != null) {
// The install half now suspends on the platform's verdict, so it
// gets its own stage — reporting "Downloading…" through it would
// be a lie once a confirm dialog is on screen.
internal.update { it.copy(installStage = InstallStage.INSTALLING) }
val outcome = installer.install(apk)
internal.update {
it.copy(installStage = outcome.stage(), installMessage = outcome.message())
}
}
}
}
private suspend fun download(apkUrl: String): File? =
runCatching { installer.downloadApk(apkUrl) }
.onFailure { e ->
internal.update {
it.copy(
installStage = InstallStage.ERROR,
installMessage = "Couldn't download update: ${ErrorCopy.fromThrowable(e)}",
)
}
}
.getOrNull()
}
@@ -1,20 +1,14 @@
package com.fabledsword.minstrel.settings.ui
import androidx.compose.foundation.layout.Arrangement
import androidx.compose.foundation.layout.Box
import androidx.compose.foundation.layout.Column
import androidx.compose.foundation.layout.Row
import androidx.compose.foundation.layout.Spacer
import androidx.compose.foundation.layout.fillMaxWidth
import androidx.compose.foundation.layout.height
import androidx.compose.foundation.layout.padding
import androidx.compose.foundation.layout.size
import androidx.compose.material3.Button
import androidx.compose.material3.CircularProgressIndicator
import androidx.compose.material3.ElevatedCard
import androidx.compose.material3.MaterialTheme
import androidx.compose.material3.OutlinedTextField
import androidx.compose.material3.Switch
import androidx.compose.material3.Text
import androidx.compose.runtime.Composable
import androidx.compose.runtime.getValue
@@ -25,44 +19,29 @@ import androidx.compose.ui.unit.dp
import androidx.hilt.navigation.compose.hiltViewModel
import androidx.lifecycle.compose.collectAsStateWithLifecycle
import com.fabledsword.minstrel.models.ListenBrainzStatus
import com.fabledsword.minstrel.theme.Muted
import com.fabledsword.minstrel.theme.Section
import com.fabledsword.minstrel.theme.SwitchRow
@Composable
fun ListenBrainzCard(viewModel: ListenBrainzViewModel = hiltViewModel()) {
val state by viewModel.state.collectAsStateWithLifecycle()
ElevatedCard(modifier = Modifier.fillMaxWidth()) {
Column(
modifier = Modifier.padding(16.dp),
verticalArrangement = Arrangement.spacedBy(8.dp),
) {
Text(
text = "ListenBrainz",
style = MaterialTheme.typography.titleMedium,
color = MaterialTheme.colorScheme.onSurface,
Section("ListenBrainz") {
if (state.isLoading) {
Box(
modifier = Modifier.fillMaxWidth().padding(16.dp),
contentAlignment = Alignment.Center,
) { CircularProgressIndicator() }
} else {
ListenBrainzForm(
status = state.status,
tokenInput = state.tokenInput,
isSaving = state.isSaving,
message = state.message,
onTokenChange = viewModel::setTokenInput,
onSaveToken = viewModel::saveToken,
onSetEnabled = viewModel::setEnabled,
)
Text(
text = "Send your plays to ListenBrainz so you can keep your " +
"scrobble history portable. Get a token at " +
"listenbrainz.org/profile — it's stored encrypted on " +
"the server and never read back.",
style = MaterialTheme.typography.bodySmall,
color = MaterialTheme.colorScheme.onSurfaceVariant,
)
if (state.isLoading) {
Box(
modifier = Modifier.fillMaxWidth().padding(16.dp),
contentAlignment = Alignment.Center,
) { CircularProgressIndicator() }
} else {
ListenBrainzForm(
status = state.status,
tokenInput = state.tokenInput,
isSaving = state.isSaving,
message = state.message,
onTokenChange = viewModel::setTokenInput,
onSaveToken = viewModel::saveToken,
onSetEnabled = viewModel::setEnabled,
)
}
}
}
}
@@ -77,25 +56,25 @@ private fun ListenBrainzForm(
onSaveToken: () -> Unit,
onSetEnabled: (Boolean) -> Unit,
) {
TokenField(
tokenInput = tokenInput,
tokenSet = status?.tokenSet == true,
onChange = onTokenChange,
val tokenSet = status?.tokenSet == true
SwitchRow(
title = "Send my plays to ListenBrainz",
summary = when {
!tokenSet -> "Save a token first"
status?.lastScrobbledAt != null -> "Last scrobble: ${status.lastScrobbledAt}"
else -> null
},
checked = status?.enabled == true,
onCheckedChange = onSetEnabled,
enabled = !isSaving && tokenSet,
)
Spacer(Modifier.height(4.dp))
TokenField(tokenInput = tokenInput, tokenSet = tokenSet, onChange = onTokenChange)
SaveTokenButton(
isSaving = isSaving,
enabled = !isSaving && tokenInput.isNotBlank(),
onClick = onSaveToken,
)
EnabledRow(status = status, isSaving = isSaving, onSetEnabled = onSetEnabled)
if (message != null) {
Text(
text = message,
style = MaterialTheme.typography.bodySmall,
color = MaterialTheme.colorScheme.onSurfaceVariant,
)
}
if (message != null) Muted(message)
}
@Composable
@@ -105,6 +84,7 @@ private fun TokenField(tokenInput: String, tokenSet: Boolean, onChange: (String)
onValueChange = onChange,
label = { Text(if (tokenSet) "Replace token" else "Token") },
placeholder = { Text(if (tokenSet) "Token saved" else "") },
supportingText = { Text("From listenbrainz.org/profile; never shown again") },
modifier = Modifier.fillMaxWidth(),
singleLine = true,
visualTransformation = PasswordVisualTransformation(),
@@ -129,36 +109,3 @@ private fun SaveTokenButton(isSaving: Boolean, enabled: Boolean, onClick: () ->
Text(if (isSaving) "Saving…" else "Save token")
}
}
@Composable
private fun EnabledRow(
status: ListenBrainzStatus?,
isSaving: Boolean,
onSetEnabled: (Boolean) -> Unit,
) {
Row(
modifier = Modifier.fillMaxWidth(),
verticalAlignment = Alignment.CenterVertically,
horizontalArrangement = Arrangement.SpaceBetween,
) {
Column(modifier = Modifier.padding(end = 12.dp)) {
Text(
text = "Send my plays to ListenBrainz",
style = MaterialTheme.typography.bodyMedium,
color = MaterialTheme.colorScheme.onSurface,
)
if (status?.lastScrobbledAt != null) {
Text(
text = "Last scrobble: ${status.lastScrobbledAt}",
style = MaterialTheme.typography.bodySmall,
color = MaterialTheme.colorScheme.onSurfaceVariant,
)
}
}
Switch(
checked = status?.enabled == true,
onCheckedChange = onSetEnabled,
enabled = !isSaving && status?.tokenSet == true,
)
}
}
@@ -1,30 +1,15 @@
package com.fabledsword.minstrel.settings.ui
import androidx.compose.foundation.layout.Arrangement
import androidx.compose.foundation.layout.Column
import androidx.compose.foundation.layout.Row
import androidx.compose.foundation.layout.Spacer
import androidx.compose.foundation.layout.fillMaxWidth
import androidx.compose.foundation.layout.padding
import androidx.compose.foundation.layout.size
import androidx.compose.material3.ElevatedCard
import androidx.compose.material3.ExperimentalMaterial3Api
import androidx.compose.material3.MaterialTheme
import androidx.compose.material3.SegmentedButton
import androidx.compose.material3.SegmentedButtonDefaults
import androidx.compose.material3.SingleChoiceSegmentedButtonRow
import androidx.compose.material3.Switch
import androidx.compose.material3.Text
import androidx.compose.runtime.Composable
import androidx.compose.runtime.getValue
import androidx.compose.ui.Alignment
import androidx.compose.ui.Modifier
import androidx.compose.ui.unit.dp
import androidx.hilt.navigation.compose.hiltViewModel
import androidx.lifecycle.compose.collectAsStateWithLifecycle
import com.fabledsword.minstrel.settings.data.NormalizationBoost
import com.fabledsword.minstrel.settings.data.NormalizationMode
import com.fabledsword.minstrel.settings.data.NormalizationPrefs
import com.fabledsword.minstrel.theme.ChoiceRow
import com.fabledsword.minstrel.theme.Section
import com.fabledsword.minstrel.theme.SwitchRow
/**
* Volume leveling. The choice is stored on the server, so the
@@ -36,98 +21,39 @@ fun NormalizationCard(viewModel: NormalizationViewModel = hiltViewModel()) {
NormalizationCardContent(prefs = prefs, onChange = viewModel::update)
}
@OptIn(ExperimentalMaterial3Api::class)
@Composable
internal fun NormalizationCardContent(
prefs: NormalizationPrefs,
onChange: (NormalizationPrefs) -> Unit,
) {
ElevatedCard(modifier = Modifier.fillMaxWidth()) {
Column(
modifier = Modifier.padding(16.dp),
verticalArrangement = Arrangement.spacedBy(8.dp),
) {
Text(
text = "Volume leveling",
style = MaterialTheme.typography.titleMedium,
color = MaterialTheme.colorScheme.onSurface,
)
Hint(modeHint(prefs.mode))
val modes = NormalizationMode.entries
SingleChoiceSegmentedButtonRow(modifier = Modifier.fillMaxWidth()) {
modes.forEachIndexed { index, mode ->
SegmentedButton(
selected = mode == prefs.mode,
onClick = { onChange(prefs.copy(mode = mode)) },
shape = SegmentedButtonDefaults.itemShape(
index = index,
count = modes.size,
),
) { Text(modeLabel(mode)) }
}
}
if (prefs.mode != NormalizationMode.OFF) {
TargetRow(prefs = prefs, onChange = onChange)
BoostRow(prefs = prefs, onChange = onChange)
}
}
}
}
@OptIn(ExperimentalMaterial3Api::class)
@Composable
private fun TargetRow(prefs: NormalizationPrefs, onChange: (NormalizationPrefs) -> Unit) {
Text(
text = "Target loudness",
style = MaterialTheme.typography.bodyLarge,
color = MaterialTheme.colorScheme.onSurface,
)
val targets = NormalizationPrefs.TARGETS
SingleChoiceSegmentedButtonRow(modifier = Modifier.fillMaxWidth()) {
targets.forEachIndexed { index, lufs ->
SegmentedButton(
selected = lufs == prefs.targetLufs,
onClick = { onChange(prefs.copy(targetLufs = lufs)) },
shape = SegmentedButtonDefaults.itemShape(
index = index,
count = targets.size,
),
) { Text("$lufs LUFS") }
}
}
}
@Composable
private fun BoostRow(prefs: NormalizationPrefs, onChange: (NormalizationPrefs) -> Unit) {
Row(verticalAlignment = Alignment.CenterVertically) {
Column(modifier = Modifier.weight(1f)) {
Text(
text = "Boost quiet tracks fully",
style = MaterialTheme.typography.bodyLarge,
color = MaterialTheme.colorScheme.onSurface,
)
Hint("A limiter catches the peaks.")
}
Spacer(Modifier.size(12.dp))
Switch(
checked = prefs.boost == NormalizationBoost.LIMITER,
onCheckedChange = { on ->
val boost = if (on) NormalizationBoost.LIMITER else NormalizationBoost.HEADROOM
onChange(prefs.copy(boost = boost))
},
Section("Volume leveling") {
ChoiceRow(
title = "Leveling",
summary = modeHint(prefs.mode),
options = NormalizationMode.entries.map { it to modeLabel(it) },
selected = prefs.mode,
onSelect = { onChange(prefs.copy(mode = it)) },
)
if (prefs.mode != NormalizationMode.OFF) {
ChoiceRow(
title = "Target loudness",
options = NormalizationPrefs.TARGETS.map { it to "$it LUFS" },
selected = prefs.targetLufs,
onSelect = { onChange(prefs.copy(targetLufs = it)) },
)
SwitchRow(
title = "Boost quiet tracks fully",
summary = "A limiter catches the peaks",
checked = prefs.boost == NormalizationBoost.LIMITER,
onCheckedChange = { on ->
val boost = if (on) NormalizationBoost.LIMITER else NormalizationBoost.HEADROOM
onChange(prefs.copy(boost = boost))
},
)
}
}
}
@Composable
private fun Hint(text: String) {
Text(
text = text,
style = MaterialTheme.typography.bodySmall,
color = MaterialTheme.colorScheme.onSurfaceVariant,
)
}
private fun modeLabel(mode: NormalizationMode): String = when (mode) {
NormalizationMode.OFF -> "Off"
NormalizationMode.AUTO -> "Auto"
@@ -1,7 +1,5 @@
package com.fabledsword.minstrel.settings.ui
import androidx.compose.foundation.layout.Arrangement
import androidx.compose.foundation.layout.Column
import androidx.compose.foundation.layout.Spacer
import androidx.compose.foundation.layout.fillMaxWidth
import androidx.compose.foundation.layout.height
@@ -10,7 +8,6 @@ import androidx.compose.foundation.layout.size
import androidx.compose.foundation.text.KeyboardOptions
import androidx.compose.material3.Button
import androidx.compose.material3.CircularProgressIndicator
import androidx.compose.material3.ElevatedCard
import androidx.compose.material3.MaterialTheme
import androidx.compose.material3.OutlinedTextField
import androidx.compose.material3.Text
@@ -22,59 +19,45 @@ import androidx.compose.ui.text.input.PasswordVisualTransformation
import androidx.compose.ui.unit.dp
import androidx.hilt.navigation.compose.hiltViewModel
import androidx.lifecycle.compose.collectAsStateWithLifecycle
import com.fabledsword.minstrel.theme.Muted
import com.fabledsword.minstrel.theme.Section
@Composable
fun PasswordCard(viewModel: PasswordViewModel = hiltViewModel()) {
val state by viewModel.state.collectAsStateWithLifecycle()
ElevatedCard(modifier = Modifier.fillMaxWidth()) {
Column(
modifier = Modifier.padding(16.dp),
verticalArrangement = Arrangement.spacedBy(8.dp),
Section("Password") {
PasswordField(
value = state.current,
label = "Current password",
onChange = viewModel::setCurrent,
)
PasswordField(
value = state.next,
label = "New password",
onChange = viewModel::setNext,
)
PasswordField(
value = state.confirm,
label = "Confirm new password",
onChange = viewModel::setConfirm,
)
Spacer(Modifier.height(4.dp))
Button(
onClick = viewModel::change,
enabled = !state.isChanging,
modifier = Modifier.fillMaxWidth(),
) {
Text(
text = "Password",
style = MaterialTheme.typography.titleMedium,
color = MaterialTheme.colorScheme.onSurface,
)
PasswordField(
value = state.current,
label = "Current password",
onChange = viewModel::setCurrent,
)
PasswordField(
value = state.next,
label = "New password",
onChange = viewModel::setNext,
)
PasswordField(
value = state.confirm,
label = "Confirm new password",
onChange = viewModel::setConfirm,
)
Spacer(Modifier.height(4.dp))
Button(
onClick = viewModel::change,
enabled = !state.isChanging,
modifier = Modifier.fillMaxWidth(),
) {
if (state.isChanging) {
CircularProgressIndicator(
modifier = Modifier.size(16.dp),
strokeWidth = 2.dp,
color = MaterialTheme.colorScheme.onPrimary,
)
Spacer(Modifier.size(8.dp))
}
Text(if (state.isChanging) "Changing…" else "Change password")
}
if (state.message != null) {
Text(
text = state.message!!,
style = MaterialTheme.typography.bodySmall,
color = MaterialTheme.colorScheme.onSurfaceVariant,
if (state.isChanging) {
CircularProgressIndicator(
modifier = Modifier.size(16.dp),
strokeWidth = 2.dp,
color = MaterialTheme.colorScheme.onPrimary,
)
Spacer(Modifier.size(8.dp))
}
Text(if (state.isChanging) "Changing…" else "Change password")
}
state.message?.let { Muted(it) }
}
}
@@ -1,8 +1,6 @@
package com.fabledsword.minstrel.settings.ui
import androidx.compose.foundation.layout.Arrangement
import androidx.compose.foundation.layout.Box
import androidx.compose.foundation.layout.Column
import androidx.compose.foundation.layout.Spacer
import androidx.compose.foundation.layout.fillMaxWidth
import androidx.compose.foundation.layout.height
@@ -10,7 +8,6 @@ import androidx.compose.foundation.layout.padding
import androidx.compose.foundation.layout.size
import androidx.compose.material3.Button
import androidx.compose.material3.CircularProgressIndicator
import androidx.compose.material3.ElevatedCard
import androidx.compose.material3.MaterialTheme
import androidx.compose.material3.OutlinedTextField
import androidx.compose.material3.Text
@@ -21,36 +18,28 @@ import androidx.compose.ui.Modifier
import androidx.compose.ui.unit.dp
import androidx.hilt.navigation.compose.hiltViewModel
import androidx.lifecycle.compose.collectAsStateWithLifecycle
import com.fabledsword.minstrel.theme.Muted
import com.fabledsword.minstrel.theme.Section
@Composable
fun ProfileCard(viewModel: ProfileViewModel = hiltViewModel()) {
val state by viewModel.state.collectAsStateWithLifecycle()
ElevatedCard(modifier = Modifier.fillMaxWidth()) {
Column(
modifier = Modifier.padding(16.dp),
verticalArrangement = Arrangement.spacedBy(8.dp),
) {
Text(
text = "Profile",
style = MaterialTheme.typography.titleMedium,
color = MaterialTheme.colorScheme.onSurface,
Section("Profile") {
if (state.isLoading) {
Box(
modifier = Modifier.fillMaxWidth().padding(16.dp),
contentAlignment = Alignment.Center,
) { CircularProgressIndicator() }
} else {
ProfileForm(
displayName = state.displayName,
email = state.email,
isSaving = state.isSaving,
message = state.message,
onDisplayNameChange = viewModel::setDisplayName,
onEmailChange = viewModel::setEmail,
onSave = viewModel::save,
)
if (state.isLoading) {
Box(
modifier = Modifier.fillMaxWidth().padding(16.dp),
contentAlignment = Alignment.Center,
) { CircularProgressIndicator() }
} else {
ProfileForm(
displayName = state.displayName,
email = state.email,
isSaving = state.isSaving,
message = state.message,
onDisplayNameChange = viewModel::setDisplayName,
onEmailChange = viewModel::setEmail,
onSave = viewModel::save,
)
}
}
}
}
@@ -95,11 +84,5 @@ private fun ProfileForm(
}
Text(if (isSaving) "Saving…" else "Save profile")
}
if (message != null) {
Text(
text = message,
style = MaterialTheme.typography.bodySmall,
color = MaterialTheme.colorScheme.onSurfaceVariant,
)
}
if (message != null) Muted(message)
}
@@ -1,34 +1,19 @@
@file:Suppress("TooManyFunctions") // Settings screen + per-card helper composables
@file:Suppress("TooManyFunctions") // Settings screen + per-section composables
package com.fabledsword.minstrel.settings.ui
import androidx.compose.foundation.clickable
import androidx.compose.foundation.rememberScrollState
import androidx.compose.foundation.verticalScroll
import androidx.compose.foundation.layout.Arrangement
import androidx.compose.foundation.layout.Box
import androidx.compose.foundation.layout.Column
import androidx.compose.foundation.layout.PaddingValues
import androidx.compose.foundation.layout.Row
import androidx.compose.foundation.layout.Spacer
import androidx.compose.foundation.layout.fillMaxSize
import androidx.compose.foundation.layout.fillMaxWidth
import androidx.compose.foundation.layout.height
import androidx.compose.foundation.layout.padding
import androidx.compose.foundation.layout.size
import androidx.compose.foundation.rememberScrollState
import androidx.compose.foundation.verticalScroll
import androidx.compose.material3.AlertDialog
import androidx.compose.material3.Button
import androidx.compose.material3.ButtonDefaults
import androidx.compose.material3.CircularProgressIndicator
import androidx.compose.material3.ElevatedCard
import androidx.compose.material3.ExperimentalMaterial3Api
import androidx.compose.material3.Icon
import androidx.compose.material3.MaterialTheme
import androidx.compose.material3.Scaffold
import androidx.compose.material3.SegmentedButton
import androidx.compose.material3.SegmentedButtonDefaults
import androidx.compose.material3.SingleChoiceSegmentedButtonRow
import androidx.compose.material3.Switch
import androidx.compose.material3.Text
import androidx.compose.material3.TextButton
import androidx.compose.runtime.Composable
@@ -37,32 +22,35 @@ import androidx.compose.runtime.getValue
import androidx.compose.runtime.mutableStateOf
import androidx.compose.runtime.remember
import androidx.compose.runtime.setValue
import androidx.compose.ui.Alignment
import androidx.compose.ui.Modifier
import androidx.compose.ui.graphics.vector.ImageVector
import androidx.compose.ui.text.style.TextOverflow
import androidx.compose.ui.unit.dp
import androidx.hilt.navigation.compose.hiltViewModel
import androidx.lifecycle.compose.collectAsStateWithLifecycle
import androidx.navigation.NavHostController
import com.composables.icons.lucide.Bell
import com.composables.icons.lucide.ChevronRight
import com.composables.icons.lucide.ListMusic
import com.composables.icons.lucide.LogOut
import com.composables.icons.lucide.Lucide
import com.composables.icons.lucide.Shield
import com.fabledsword.minstrel.BuildConfig
import com.fabledsword.minstrel.nav.Admin
import com.fabledsword.minstrel.nav.NotificationSettings
import com.fabledsword.minstrel.nav.Requests
import com.fabledsword.minstrel.nav.Settings as SettingsRoute
import com.fabledsword.minstrel.nav.ServerUrl
import com.fabledsword.minstrel.shared.widgets.ShellContentWindowInsets
import com.fabledsword.minstrel.nav.Settings as SettingsRoute
import com.fabledsword.minstrel.shared.widgets.MinstrelTopAppBar
import com.fabledsword.minstrel.shared.widgets.ShellContentWindowInsets
import com.fabledsword.minstrel.theme.ChoiceRow
import com.fabledsword.minstrel.theme.RowChevron
import com.fabledsword.minstrel.theme.RowSpinner
import com.fabledsword.minstrel.theme.Section
import com.fabledsword.minstrel.theme.SectionEdge
import com.fabledsword.minstrel.theme.SectionGap
import com.fabledsword.minstrel.theme.SettingRow
import com.fabledsword.minstrel.theme.SwitchRow
import com.fabledsword.minstrel.theme.ThemeMode
import com.fabledsword.minstrel.theme.ThemePreferenceViewModel
import com.fabledsword.minstrel.update.data.InstallStage
import com.fabledsword.minstrel.update.data.isBusy
import com.fabledsword.minstrel.update.ui.AppVersionCard
import com.fabledsword.minstrel.update.ui.CheckWhileShown
import com.fabledsword.minstrel.update.ui.UpdateViewModel
@Composable
fun SettingsScreen(
@@ -119,7 +107,7 @@ fun SettingsScreen(
@Composable
private fun SettingsList(
inner: androidx.compose.foundation.layout.PaddingValues,
inner: PaddingValues,
state: SettingsState,
themeMode: ThemeMode,
onPickTheme: (ThemeMode) -> Unit,
@@ -134,48 +122,26 @@ private fun SettingsList(
.fillMaxSize()
.padding(inner)
.verticalScroll(rememberScrollState())
.padding(16.dp),
verticalArrangement = Arrangement.spacedBy(16.dp),
.padding(horizontal = SectionEdge, vertical = SectionEdge),
verticalArrangement = Arrangement.spacedBy(SectionGap),
) {
AccountCard(
username = state.username,
isAdmin = state.isAdmin,
serverUrl = state.serverUrl,
AccountSection(
state = state,
onNavToRequests = onNavToRequests,
onNavToNotifications = onNavToNotifications,
onNavToAdmin = onNavToAdmin,
onSignOutClick = onSignOutClick,
)
NavTile(
icon = Lucide.ListMusic,
title = "My requests",
subtitle = "Track what you've asked Minstrel to add",
onClick = onNavToRequests,
)
NavTile(
icon = Lucide.Bell,
title = "Notifications",
subtitle = "What reaches you, and where",
onClick = onNavToNotifications,
)
if (state.isAdmin) {
NavTile(
icon = Lucide.Shield,
title = "Admin",
subtitle = "Manage requests, quarantine, and users",
onClick = onNavToAdmin,
)
}
ProfileCard()
PasswordCard()
ListenBrainzCard()
if (state.diagnosticsEnabledByAdmin) {
DiagnosticsCard(
optOut = state.diagnosticsOptOut,
onToggle = onToggleDiagnostics,
)
DiagnosticsSection(optOut = state.diagnosticsOptOut, onToggle = onToggleDiagnostics)
}
NormalizationCard()
AppearanceCard(themeMode = themeMode, onPick = onPickTheme)
AppearanceSection(themeMode = themeMode, onPick = onPickTheme)
StorageCard()
AboutCard()
SignOutButton(isSigningOut = state.isSigningOut, onSignOut = onSignOutClick)
}
}
@@ -205,84 +171,48 @@ private fun SignOutConfirmDialog(onConfirm: () -> Unit, onDismiss: () -> Unit) {
)
}
/** Who is signed in, where, the screens that belong to the account, and signing out. */
@Composable
private fun NavTile(
icon: ImageVector,
title: String,
subtitle: String,
onClick: () -> Unit,
private fun AccountSection(
state: SettingsState,
onNavToRequests: () -> Unit,
onNavToNotifications: () -> Unit,
onNavToAdmin: () -> Unit,
onSignOutClick: () -> Unit,
) {
ElevatedCard(
modifier = Modifier
.fillMaxWidth()
.clickable(onClick = onClick),
) {
Row(
modifier = Modifier.padding(horizontal = 16.dp, vertical = 12.dp),
verticalAlignment = Alignment.CenterVertically,
) {
Icon(
imageVector = icon,
contentDescription = null,
tint = MaterialTheme.colorScheme.onSurface,
)
Spacer(Modifier.size(16.dp))
Column(modifier = Modifier.weight(1f)) {
Text(
text = title,
style = MaterialTheme.typography.titleMedium,
color = MaterialTheme.colorScheme.onSurface,
)
Text(
text = subtitle,
style = MaterialTheme.typography.bodySmall,
color = MaterialTheme.colorScheme.onSurfaceVariant,
)
}
Icon(
imageVector = Lucide.ChevronRight,
contentDescription = null,
tint = MaterialTheme.colorScheme.onSurfaceVariant,
)
}
}
}
@Composable
private fun AccountCard(username: String, isAdmin: Boolean, serverUrl: String) {
ElevatedCard(modifier = Modifier.fillMaxWidth()) {
Column(
modifier = Modifier.padding(16.dp),
verticalArrangement = Arrangement.spacedBy(4.dp),
) {
Text(
text = "Account",
style = MaterialTheme.typography.titleMedium,
color = MaterialTheme.colorScheme.onSurface,
)
val adminTag = if (isAdmin) " · admin" else ""
Text(
text = when {
username.isEmpty() -> "Signed in"
else -> "Signed in as $username$adminTag"
},
style = MaterialTheme.typography.bodyMedium,
color = MaterialTheme.colorScheme.onSurfaceVariant,
)
Spacer(Modifier.height(8.dp))
Text(
text = "Server",
style = MaterialTheme.typography.titleSmall,
color = MaterialTheme.colorScheme.onSurface,
)
Text(
text = serverUrl,
style = MaterialTheme.typography.bodySmall,
color = MaterialTheme.colorScheme.onSurfaceVariant,
maxLines = 1,
overflow = TextOverflow.Ellipsis,
)
Section("Account") {
SettingRow(
title = state.username.ifEmpty { "Signed in" },
summary = if (state.isAdmin) "Signed in · admin" else "Signed in",
)
SettingRow(title = "Server", summary = state.serverUrl)
SettingRow(
title = "My requests",
summary = "What you've asked Minstrel to add",
icon = Lucide.ListMusic,
onClick = onNavToRequests,
) { RowChevron() }
SettingRow(
title = "Notifications",
summary = "What reaches you, and where",
icon = Lucide.Bell,
onClick = onNavToNotifications,
) { RowChevron() }
if (state.isAdmin) {
SettingRow(
title = "Admin",
summary = "Requests, quarantine and users",
icon = Lucide.Shield,
onClick = onNavToAdmin,
) { RowChevron() }
}
SettingRow(
title = if (state.isSigningOut) "Signing out…" else "Sign out",
icon = Lucide.LogOut,
tone = MaterialTheme.colorScheme.error,
enabled = !state.isSigningOut,
onClick = onSignOutClick,
) { if (state.isSigningOut) RowSpinner() }
}
}
@@ -290,74 +220,27 @@ private fun AccountCard(username: String, isAdmin: Boolean, serverUrl: String) {
// the user turn reporting OFF on this device (battery/privacy) without
// touching the account flag. Checked = reporting on = NOT opted out.
@Composable
private fun DiagnosticsCard(optOut: Boolean, onToggle: (Boolean) -> Unit) {
ElevatedCard(modifier = Modifier.fillMaxWidth()) {
Column(
modifier = Modifier.padding(16.dp),
verticalArrangement = Arrangement.spacedBy(8.dp),
) {
Text(
text = "Diagnostics",
style = MaterialTheme.typography.titleMedium,
color = MaterialTheme.colorScheme.onSurface,
)
Row(verticalAlignment = Alignment.CenterVertically) {
Column(modifier = Modifier.weight(1f)) {
Text(
text = "Send diagnostic reports",
style = MaterialTheme.typography.bodyLarge,
color = MaterialTheme.colorScheme.onSurface,
)
Text(
text = "Your admin enabled debug mode to investigate an " +
"issue. This device streams connectivity and playback " +
"diagnostics. Turn off any time.",
style = MaterialTheme.typography.bodySmall,
color = MaterialTheme.colorScheme.onSurfaceVariant,
)
}
Spacer(Modifier.size(12.dp))
Switch(
checked = !optOut,
onCheckedChange = { checked -> onToggle(!checked) },
)
}
}
private fun DiagnosticsSection(optOut: Boolean, onToggle: (Boolean) -> Unit) {
Section("Diagnostics") {
SwitchRow(
title = "Send diagnostic reports",
summary = "Your admin turned on debug mode",
checked = !optOut,
onCheckedChange = { checked -> onToggle(!checked) },
)
}
}
@OptIn(ExperimentalMaterial3Api::class)
@Composable
private fun AppearanceCard(themeMode: ThemeMode, onPick: (ThemeMode) -> Unit) {
val options = ThemeMode.entries
ElevatedCard(modifier = Modifier.fillMaxWidth()) {
Column(
modifier = Modifier.padding(16.dp),
verticalArrangement = Arrangement.spacedBy(8.dp),
) {
Text(
text = "Appearance",
style = MaterialTheme.typography.titleMedium,
color = MaterialTheme.colorScheme.onSurface,
)
Text(
text = "System follows your device setting.",
style = MaterialTheme.typography.bodySmall,
color = MaterialTheme.colorScheme.onSurfaceVariant,
)
SingleChoiceSegmentedButtonRow(modifier = Modifier.fillMaxWidth()) {
options.forEachIndexed { index, mode ->
SegmentedButton(
selected = mode == themeMode,
onClick = { onPick(mode) },
shape = SegmentedButtonDefaults.itemShape(
index = index,
count = options.size,
),
) { Text(displayName(mode)) }
}
}
}
private fun AppearanceSection(themeMode: ThemeMode, onPick: (ThemeMode) -> Unit) {
Section("Appearance") {
ChoiceRow(
title = "Theme",
summary = if (themeMode == ThemeMode.SYSTEM) "Follows your device" else null,
options = ThemeMode.entries.map { it to displayName(it) },
selected = themeMode,
onSelect = onPick,
)
}
}
@@ -367,136 +250,12 @@ private fun displayName(mode: ThemeMode): String = when (mode) {
ThemeMode.DARK -> "Dark"
}
/**
* The About card is the app-version card Home's update sheet holds too; on
* Settings it also asks the server again while the screen is up.
*/
@Composable
private fun AboutCard(viewModel: AboutCardViewModel = hiltViewModel()) {
val state by viewModel.state.collectAsStateWithLifecycle()
ElevatedCard(modifier = Modifier.fillMaxWidth()) {
Column(
modifier = Modifier.padding(16.dp),
verticalArrangement = Arrangement.spacedBy(8.dp),
) {
Text(
text = "About",
style = MaterialTheme.typography.titleMedium,
color = MaterialTheme.colorScheme.onSurface,
)
Text(
text = "Installed: Minstrel ${state.installedVersion} " +
"(${BuildConfig.VERSION_CODE})",
style = MaterialTheme.typography.bodySmall,
color = MaterialTheme.colorScheme.onSurfaceVariant,
)
UpdateControls(state = state, viewModel = viewModel)
}
}
}
@Composable
private fun UpdateControls(state: AboutUiState, viewModel: AboutCardViewModel) {
if (!state.selfUpdates) {
Text(
text = "Debug build: updates install from Android Studio.",
style = MaterialTheme.typography.bodySmall,
color = MaterialTheme.colorScheme.onSurfaceVariant,
)
return
}
UpdateCheckLine(result = state.result)
Button(
onClick = viewModel::checkForUpdates,
enabled = !state.isChecking && !state.installStage.isBusy(),
modifier = Modifier.fillMaxWidth(),
) {
if (state.isChecking) {
ButtonSpinner()
}
Text(if (state.isChecking) "Checking…" else "Check for updates")
}
val available = state.result as? UpdateCheckResult.UpdateAvailable
if (available != null) {
InstallButton(
version = available.info.version,
stage = state.installStage,
onClick = { viewModel.install(available.info) },
)
}
if (state.installMessage != null) {
Text(
text = state.installMessage,
style = MaterialTheme.typography.bodySmall,
color = MaterialTheme.colorScheme.onSurfaceVariant,
)
}
}
@Composable
private fun InstallButton(version: String, stage: InstallStage, onClick: () -> Unit) {
Button(
onClick = onClick,
enabled = !stage.isBusy(),
modifier = Modifier.fillMaxWidth(),
) {
if (stage.isBusy()) {
ButtonSpinner()
}
Text(
when (stage) {
InstallStage.DOWNLOADING -> "Downloading…"
InstallStage.INSTALLING -> "Installing…"
else -> "Install $version"
},
)
}
}
@Composable
private fun ButtonSpinner() {
CircularProgressIndicator(
modifier = Modifier.size(16.dp),
strokeWidth = 2.dp,
color = MaterialTheme.colorScheme.onPrimary,
)
Spacer(Modifier.size(8.dp))
}
@Composable
private fun UpdateCheckLine(result: UpdateCheckResult) {
val text = when (result) {
UpdateCheckResult.Idle -> return
UpdateCheckResult.Latest -> "You're on the latest version."
is UpdateCheckResult.UpdateAvailable ->
"Update available: ${result.info.version}"
is UpdateCheckResult.Error -> "Couldn't check: ${result.message}"
}
Text(
text = text,
style = MaterialTheme.typography.bodySmall,
color = MaterialTheme.colorScheme.onSurfaceVariant,
)
}
@Composable
private fun SignOutButton(isSigningOut: Boolean, onSignOut: () -> Unit) {
Button(
onClick = onSignOut,
enabled = !isSigningOut,
modifier = Modifier.fillMaxWidth(),
colors = ButtonDefaults.buttonColors(
containerColor = MaterialTheme.colorScheme.errorContainer,
contentColor = MaterialTheme.colorScheme.onErrorContainer,
),
) {
if (isSigningOut) {
CircularProgressIndicator(
modifier = Modifier.size(20.dp),
color = MaterialTheme.colorScheme.onErrorContainer,
strokeWidth = 2.dp,
)
} else {
Box(modifier = Modifier.padding(end = 4.dp)) {
Icon(Lucide.LogOut, contentDescription = null)
}
Text("Sign out")
}
}
private fun AboutCard(model: UpdateViewModel = hiltViewModel()) {
CheckWhileShown(model)
AppVersionCard(model)
}
@@ -1,23 +1,9 @@
package com.fabledsword.minstrel.settings.ui
import androidx.compose.foundation.layout.Arrangement
import androidx.compose.foundation.layout.Column
import androidx.compose.foundation.layout.Row
import androidx.compose.foundation.layout.Spacer
import androidx.compose.foundation.layout.fillMaxWidth
import androidx.compose.foundation.layout.padding
import androidx.compose.foundation.layout.size
import androidx.compose.foundation.selection.toggleable
import androidx.compose.material3.AlertDialog
import androidx.compose.material3.Button
import androidx.compose.material3.ButtonDefaults
import androidx.compose.material3.CircularProgressIndicator
import androidx.compose.material3.DropdownMenu
import androidx.compose.material3.DropdownMenuItem
import androidx.compose.material3.ElevatedCard
import androidx.compose.material3.MaterialTheme
import androidx.compose.material3.OutlinedButton
import androidx.compose.material3.Switch
import androidx.compose.material3.Text
import androidx.compose.material3.TextButton
import androidx.compose.runtime.Composable
@@ -25,12 +11,13 @@ import androidx.compose.runtime.getValue
import androidx.compose.runtime.mutableStateOf
import androidx.compose.runtime.remember
import androidx.compose.runtime.setValue
import androidx.compose.ui.Alignment
import androidx.compose.ui.Modifier
import androidx.compose.ui.semantics.Role
import androidx.compose.ui.unit.dp
import androidx.hilt.navigation.compose.hiltViewModel
import androidx.lifecycle.compose.collectAsStateWithLifecycle
import com.fabledsword.minstrel.theme.ChoiceRow
import com.fabledsword.minstrel.theme.RowSpinner
import com.fabledsword.minstrel.theme.Section
import com.fabledsword.minstrel.theme.SettingRow
import com.fabledsword.minstrel.theme.SwitchRow
private const val GIB: Long = 1024L * 1024 * 1024
private const val MIB: Long = 1024L * 1024
@@ -48,44 +35,39 @@ private val CAP_OPTIONS: List<Pair<Long, String>> = listOf(
fun StorageCard(viewModel: StorageViewModel = hiltViewModel()) {
val state by viewModel.state.collectAsStateWithLifecycle()
var showClearConfirm by remember { mutableStateOf(false) }
ElevatedCard(modifier = Modifier.fillMaxWidth()) {
Column(
modifier = Modifier.padding(16.dp),
verticalArrangement = Arrangement.spacedBy(8.dp),
) {
Text(
text = "Storage",
style = MaterialTheme.typography.titleMedium,
color = MaterialTheme.colorScheme.onSurface,
)
// Two budgets: a liked track that is also recently played counts
// against Liked songs only. The prefetch window has no control;
// AudioPrefetcher uses the stored value (default 5).
CapDropdown(
label = "Liked songs",
hint = "${formatBytes(state.likedBytes)} used",
value = state.settings.likedCapBytes,
onSet = viewModel::setLikedCapBytes,
)
CapDropdown(
label = "Recently played",
hint = "${formatBytes(state.recentBytes)} used",
value = state.settings.rollingCapBytes,
onSet = viewModel::setRollingCapBytes,
)
SwitchRow(
label = "Keep liked songs offline",
hint = "Downloads on Wi-Fi",
checked = state.settings.cacheLikedTracks,
onCheckedChange = viewModel::setCacheLikedTracks,
)
StorageActions(
isSyncing = state.isSyncing,
isClearing = state.isClearing,
onSync = viewModel::syncNow,
onClear = { showClearConfirm = true },
)
}
Section("Storage") {
// Two budgets: a liked track that is also recently played counts
// against Liked songs only. The prefetch window has no control;
// AudioPrefetcher uses the stored value (default 5).
CapRow(
title = "Liked songs",
used = state.likedBytes,
value = state.settings.likedCapBytes,
onSet = viewModel::setLikedCapBytes,
)
CapRow(
title = "Recently played",
used = state.recentBytes,
value = state.settings.rollingCapBytes,
onSet = viewModel::setRollingCapBytes,
)
SwitchRow(
title = "Keep liked songs offline",
summary = "Downloads on Wi-Fi",
checked = state.settings.cacheLikedTracks,
onCheckedChange = viewModel::setCacheLikedTracks,
)
SettingRow(
title = if (state.isSyncing) "Syncing…" else "Sync library now",
enabled = !state.isSyncing,
onClick = viewModel::syncNow,
) { if (state.isSyncing) RowSpinner() }
SettingRow(
title = if (state.isClearing) "Clearing…" else "Clear cached audio",
tone = MaterialTheme.colorScheme.error,
enabled = !state.isClearing,
onClick = { showClearConfirm = true },
) { if (state.isClearing) RowSpinner() }
}
if (showClearConfirm) {
ClearCacheConfirmDialog(
@@ -98,128 +80,19 @@ fun StorageCard(viewModel: StorageViewModel = hiltViewModel()) {
}
}
/** A storage budget: how much it holds now, and the cap it may grow to. */
@Composable
private fun CapDropdown(label: String, hint: String, value: Long, onSet: (Long) -> Unit) {
val selectedLabel = CAP_OPTIONS.firstOrNull { it.first == value }?.second
?: formatBytes(value)
LabeledDropdown(
label = label,
hint = hint,
selectedLabel = selectedLabel,
private fun CapRow(title: String, used: Long, value: Long, onSet: (Long) -> Unit) {
ChoiceRow(
title = title,
summary = "${formatBytes(used)} used",
options = CAP_OPTIONS,
selected = value,
onSelect = onSet,
selectedLabel = CAP_OPTIONS.firstOrNull { it.first == value }?.second ?: formatBytes(value),
)
}
/** A setting's name, with one short line under it. */
@Composable
private fun RowLabel(label: String, hint: String, modifier: Modifier = Modifier) {
Column(modifier = modifier) {
Text(
text = label,
style = MaterialTheme.typography.bodyMedium,
color = MaterialTheme.colorScheme.onSurface,
)
Text(
text = hint,
style = MaterialTheme.typography.bodySmall,
color = MaterialTheme.colorScheme.onSurfaceVariant,
)
}
}
@Composable
private fun SwitchRow(label: String, hint: String, checked: Boolean, onCheckedChange: (Boolean) -> Unit) {
Row(
modifier = Modifier
.fillMaxWidth()
.toggleable(value = checked, role = Role.Switch, onValueChange = onCheckedChange),
verticalAlignment = Alignment.CenterVertically,
) {
RowLabel(label = label, hint = hint, modifier = Modifier.weight(1f))
Spacer(Modifier.size(12.dp))
Switch(checked = checked, onCheckedChange = null)
}
}
/**
* Label and hint on the left, button-with-current-value on the right that
* opens a DropdownMenu of [options]. Used by the cap selectors.
*/
@Composable
private fun <T> LabeledDropdown(
label: String,
hint: String,
selectedLabel: String,
options: List<Pair<T, String>>,
onSelect: (T) -> Unit,
) {
var expanded by remember { mutableStateOf(false) }
Row(
modifier = Modifier.fillMaxWidth(),
verticalAlignment = Alignment.CenterVertically,
) {
RowLabel(label = label, hint = hint, modifier = Modifier.weight(1f))
OutlinedButton(onClick = { expanded = true }) {
Text(selectedLabel)
DropdownMenu(
expanded = expanded,
onDismissRequest = { expanded = false },
) {
options.forEach { (value, name) ->
DropdownMenuItem(
text = { Text(name) },
onClick = {
expanded = false
onSelect(value)
},
)
}
}
}
}
}
@Composable
private fun StorageActions(
isSyncing: Boolean,
isClearing: Boolean,
onSync: () -> Unit,
onClear: () -> Unit,
) {
Row(
modifier = Modifier.fillMaxWidth(),
horizontalArrangement = Arrangement.spacedBy(8.dp),
) {
OutlinedButton(
onClick = onClear,
enabled = !isClearing,
) {
if (isClearing) {
CircularProgressIndicator(
modifier = Modifier.size(16.dp),
strokeWidth = 2.dp,
)
Spacer(Modifier.size(8.dp))
}
Text(if (isClearing) "Clearing…" else "Clear cache")
}
OutlinedButton(
onClick = onSync,
enabled = !isSyncing,
) {
if (isSyncing) {
CircularProgressIndicator(
modifier = Modifier.size(16.dp),
strokeWidth = 2.dp,
)
Spacer(Modifier.size(8.dp))
}
Text(if (isSyncing) "Syncing…" else "Sync now")
}
}
}
@Composable
private fun ClearCacheConfirmDialog(onConfirm: () -> Unit, onDismiss: () -> Unit) {
AlertDialog(
@@ -246,7 +119,7 @@ private fun ClearCacheConfirmDialog(onConfirm: () -> Unit, onDismiss: () -> Unit
)
}
private fun formatBytes(n: Long): String {
internal fun formatBytes(n: Long): String {
if (n <= 0L) return "0 B"
return when {
n < KIB -> "$n B"
@@ -21,7 +21,6 @@ import com.fabledsword.minstrel.cache.mutations.OfflineWriteHintViewModel
import com.fabledsword.minstrel.connectivity.ui.ConnectionErrorBanner
import com.fabledsword.minstrel.player.ui.MiniPlayer
import com.fabledsword.minstrel.player.ui.PlaybackErrorViewModel
import com.fabledsword.minstrel.update.ui.UpdateBanner
import com.fabledsword.minstrel.update.ui.VersionTooOldBanner
import com.fabledsword.minstrel.shared.widgets.trackactions.TrackActionsViewModel
@@ -63,7 +62,7 @@ val ShellContentWindowInsets: WindowInsets = WindowInsets(0, 0, 0, 0)
* [onExpandPlayer] is the navigate-to-NowPlaying callback passed
* through to MiniPlayer's tap-handler. No NavHostController parameter
* because nothing inside the shell currently navigates - if a banner
* (UpdateBanner / VersionTooOldBanner / ConnectionErrorBanner) ever
* (VersionTooOldBanner / ConnectionErrorBanner) ever
* grows tap-to-navigate, plumb a NavHostController in at that point.
*/
@Composable
@@ -97,12 +96,10 @@ fun ShellScaffold(
// the status bar — no double inset, and no banner drawing under it.
Column(modifier = modifier.fillMaxSize().statusBarsPadding()) {
// Banner slot. VersionTooOldBanner appears when the server's
// min_client_version exceeds our build; UpdateBanner softly
// nudges an available (newer-than-installed) APK; and
// ConnectionErrorBanner shows when the device loses usable
// internet.
// min_client_version exceeds our build, and ConnectionErrorBanner
// shows when the device loses usable internet. A newer app is not
// a banner: Home tells of it in a snackbar (UpdateNotices).
VersionTooOldBanner()
UpdateBanner()
ConnectionErrorBanner()
// The window is edge-to-edge, so something has to own the
// bottom navigation-bar inset, and the shell claims it once
@@ -0,0 +1,262 @@
@file:Suppress("TooManyFunctions") // the settings vocabulary: a plate and its row kinds
package com.fabledsword.minstrel.theme
import androidx.compose.foundation.clickable
import androidx.compose.foundation.layout.Arrangement
import androidx.compose.foundation.layout.Box
import androidx.compose.foundation.layout.Column
import androidx.compose.foundation.layout.ColumnScope
import androidx.compose.foundation.layout.Row
import androidx.compose.foundation.layout.RowScope
import androidx.compose.foundation.layout.Spacer
import androidx.compose.foundation.layout.fillMaxWidth
import androidx.compose.foundation.layout.height
import androidx.compose.foundation.layout.heightIn
import androidx.compose.foundation.layout.padding
import androidx.compose.foundation.layout.size
import androidx.compose.foundation.selection.toggleable
import androidx.compose.foundation.shape.RoundedCornerShape
import androidx.compose.material3.CircularProgressIndicator
import androidx.compose.material3.DropdownMenu
import androidx.compose.material3.DropdownMenuItem
import androidx.compose.material3.Icon
import androidx.compose.material3.MaterialTheme
import androidx.compose.material3.Surface
import androidx.compose.material3.Switch
import androidx.compose.material3.Text
import androidx.compose.runtime.Composable
import androidx.compose.runtime.getValue
import androidx.compose.runtime.mutableStateOf
import androidx.compose.runtime.remember
import androidx.compose.runtime.setValue
import androidx.compose.ui.Alignment
import androidx.compose.ui.Modifier
import androidx.compose.ui.draw.alpha
import androidx.compose.ui.draw.clip
import androidx.compose.ui.graphics.Color
import androidx.compose.ui.graphics.vector.ImageVector
import androidx.compose.ui.semantics.Role
import androidx.compose.ui.unit.dp
import com.composables.icons.lucide.ChevronRight
import com.composables.icons.lucide.ChevronDown
import com.composables.icons.lucide.Lucide
/**
* The plate every group of settings sits on. Every section gets the same
* plate, so the screen reads as one thing; a section that matters earns its
* weight with what is inside it, never with a louder box.
*
* The rhythm between lines lives here, not at the call sites.
*/
@Composable
fun Section(
title: String,
modifier: Modifier = Modifier,
trailing: @Composable (RowScope.() -> Unit)? = null,
content: @Composable ColumnScope.() -> Unit,
) {
Surface(
modifier = modifier.fillMaxWidth(),
shape = RoundedCornerShape(FabledSwordFlatTokens.radiusLg),
// `iron`, the raised surface, on the `obsidian` ground.
color = MaterialTheme.colorScheme.surface,
contentColor = MaterialTheme.colorScheme.onSurface,
) {
Column(
modifier = Modifier.padding(SectionInset),
verticalArrangement = Arrangement.spacedBy(SectionRow),
) {
Row(
modifier = Modifier.fillMaxWidth(),
verticalAlignment = Alignment.CenterVertically,
horizontalArrangement = Arrangement.SpaceBetween,
) {
Text(title, style = MaterialTheme.typography.titleMedium)
if (trailing != null) {
Row(verticalAlignment = Alignment.CenterVertically, content = trailing)
}
}
Spacer(Modifier.height(TITLE_GAP - SectionRow))
content()
}
}
}
/** The quiet line: a caption, a count, a consequence. */
@Composable
fun Muted(said: String, modifier: Modifier = Modifier, tone: Color = Color.Unspecified) {
Text(
said,
style = MaterialTheme.typography.bodySmall,
color = if (tone == Color.Unspecified) MaterialTheme.colorScheme.onSurfaceVariant else tone,
modifier = modifier,
)
}
/**
* One line of a settings section: what it is, one short line about it, and
* its control at the end. A settings screen is a menu, so the summary is one
* short line, never an explanation.
*
* With [onClick] the whole row is the tap target. [tone] colours the title of
* a row reporting something wrong, so the control beside it still reads as a
* control.
*/
@Composable
fun SettingRow(
title: String,
modifier: Modifier = Modifier,
summary: String? = null,
icon: ImageVector? = null,
tone: Color = Color.Unspecified,
enabled: Boolean = true,
onClick: (() -> Unit)? = null,
trailing: @Composable (RowScope.() -> Unit)? = null,
) {
val tappable = if (onClick != null) {
Modifier
.clip(RoundedCornerShape(FabledSwordFlatTokens.radiusMd))
.clickable(enabled = enabled, role = Role.Button, onClick = onClick)
} else {
Modifier
}
Row(
modifier = modifier
.fillMaxWidth()
.heightIn(min = ROW_HEIGHT)
.then(tappable)
.alpha(if (enabled) 1f else DISABLED_ALPHA),
verticalAlignment = Alignment.CenterVertically,
) {
if (icon != null) {
Icon(
icon,
contentDescription = null,
tint = if (tone == Color.Unspecified) MaterialTheme.colorScheme.onSurfaceVariant else tone,
modifier = Modifier.padding(end = ICON_GAP).size(ICON),
)
}
Column(Modifier.weight(1f).padding(vertical = ROW_PAD)) {
Text(
title,
style = MaterialTheme.typography.bodyLarge,
color = if (tone == Color.Unspecified) MaterialTheme.colorScheme.onSurface else tone,
)
if (summary != null) Muted(summary)
}
if (trailing != null) {
Row(
modifier = Modifier.padding(start = ICON_GAP),
verticalAlignment = Alignment.CenterVertically,
content = trailing,
)
}
}
}
/** A row whose control is a switch; the whole row toggles it. */
@Composable
fun SwitchRow(
title: String,
checked: Boolean,
onCheckedChange: (Boolean) -> Unit,
modifier: Modifier = Modifier,
summary: String? = null,
enabled: Boolean = true,
) {
SettingRow(
title = title,
summary = summary,
enabled = enabled,
modifier = modifier
.clip(RoundedCornerShape(FabledSwordFlatTokens.radiusMd))
.toggleable(value = checked, enabled = enabled, role = Role.Switch, onValueChange = onCheckedChange),
) {
Switch(checked = checked, onCheckedChange = null, enabled = enabled)
}
}
/**
* A one-of-N setting: the row shows the current choice, and a tap opens the
* choices under it.
*/
@Composable
fun <T> ChoiceRow(
title: String,
options: List<Pair<T, String>>,
selected: T,
onSelect: (T) -> Unit,
modifier: Modifier = Modifier,
summary: String? = null,
selectedLabel: String = options.firstOrNull { it.first == selected }?.second.orEmpty(),
) {
var open by remember { mutableStateOf(false) }
Box(modifier) {
SettingRow(title = title, summary = summary, onClick = { open = true }) {
RowValue(selectedLabel)
Icon(
Lucide.ChevronDown,
contentDescription = null,
tint = MaterialTheme.colorScheme.onSurfaceVariant,
modifier = Modifier.padding(start = 4.dp).size(SMALL_ICON),
)
}
DropdownMenu(expanded = open, onDismissRequest = { open = false }) {
options.forEach { (value, name) ->
DropdownMenuItem(
text = { Text(name) },
onClick = {
open = false
onSelect(value)
},
)
}
}
}
}
/** A row's value, set to the right: a count, a version, a size. */
@Composable
fun RowValue(said: String) {
Text(
said,
style = MaterialTheme.typography.bodyMedium,
color = MaterialTheme.colorScheme.onSurfaceVariant,
)
}
/** Work in progress, at the size a row's trailing slot expects. */
@Composable
fun RowSpinner() {
CircularProgressIndicator(modifier = Modifier.size(SPINNER), strokeWidth = 2.dp)
}
/** "This opens something", for a row whose tap leads somewhere. */
@Composable
fun RowChevron() {
Icon(Lucide.ChevronRight, contentDescription = null, tint = MaterialTheme.colorScheme.onSurfaceVariant)
}
/** The screen margin either side of a plate. */
val SectionEdge = 16.dp
/** Between one plate and the next. */
val SectionGap = 14.dp
/** Inside a plate. */
val SectionInset = 18.dp
/** Between one line of a plate and the next. */
val SectionRow = 6.dp
private val TITLE_GAP = 12.dp
/** Material's one-line list item height: the smallest a thumb reliably hits. */
private val ROW_HEIGHT = 48.dp
private val ROW_PAD = 6.dp
private val ICON = 22.dp
private val SMALL_ICON = 16.dp
private val ICON_GAP = 14.dp
private const val DISABLED_ALPHA = 0.5f
private val SPINNER = 16.dp
@@ -6,24 +6,13 @@ import android.net.Uri
import android.os.Build
import android.provider.Settings
import dagger.hilt.android.qualifiers.ApplicationContext
import kotlinx.coroutines.Dispatchers
import kotlinx.coroutines.withContext
import okhttp3.OkHttpClient
import okhttp3.Request
import java.io.File
import java.io.IOException
import javax.inject.Inject
import javax.inject.Singleton
private const val APK_FILENAME = "minstrel-update.apk"
/**
* Downloads the server-bundled APK and installs it over ourselves.
*
* The download goes through the shared [OkHttpClient] so it inherits
* the auth cookie + the BaseUrlInterceptor host rewrite (apkUrl is
* server-relative, e.g. `/api/client/apk`). The APK lands in the
* cache dir; [SelfUpdateSession] streams it from there into a
* Installs a downloaded APK over ourselves. [UpdateRepository] brings it
* down; [SelfUpdateSession] streams it from there into a
* [android.content.pm.PackageInstaller] session.
*
* On Android O+ the user must have granted "install unknown apps"
@@ -34,26 +23,8 @@ private const val APK_FILENAME = "minstrel-update.apk"
@Singleton
class ApkInstaller @Inject constructor(
@ApplicationContext private val context: Context,
private val okHttpClient: OkHttpClient,
private val session: SelfUpdateSession,
) {
suspend fun downloadApk(apkUrl: String): File = withContext(Dispatchers.IO) {
val request = Request.Builder()
.url("http://placeholder.invalid$apkUrl")
.build()
okHttpClient.newCall(request).execute().use { response ->
if (!response.isSuccessful) {
throw IOException("Download failed: HTTP ${response.code}")
}
val body = response.body ?: throw IOException("Empty download body")
val file = File(context.cacheDir, APK_FILENAME)
body.byteStream().use { input ->
file.outputStream().use { output -> input.copyTo(output) }
}
file
}
}
/** True when the system will accept an install intent without a permission detour. */
fun canInstall(): Boolean =
Build.VERSION.SDK_INT < Build.VERSION_CODES.O ||
@@ -0,0 +1,196 @@
package com.fabledsword.minstrel.update.data
import com.fabledsword.minstrel.di.ApplicationScope
import com.fabledsword.minstrel.models.UpdateInfo
import kotlinx.coroutines.CoroutineScope
import kotlinx.coroutines.Job
import kotlinx.coroutines.channels.Channel
import kotlinx.coroutines.flow.Flow
import kotlinx.coroutines.flow.MutableStateFlow
import kotlinx.coroutines.flow.StateFlow
import kotlinx.coroutines.flow.asStateFlow
import kotlinx.coroutines.flow.receiveAsFlow
import kotlinx.coroutines.flow.update
import kotlinx.coroutines.launch
import java.io.File
import javax.inject.Inject
import javax.inject.Singleton
data class UpdateUiState(
/** Null when there is nothing newer to offer. */
val offer: UpdateInfo? = null,
val stage: InstallStage = InstallStage.IDLE,
val progress: Float = 0f,
val problem: String? = null,
/** Asking the server which app it offers. */
val checking: Boolean = false,
/** Why the last ask got no answer, when it did not. */
val unchecked: String? = null,
/** The offer is already downloaded and checked: Install is all that is left. */
val ready: Boolean = false,
)
/** A newer build to tell the listener of, and whether it is already here to install. */
data class UpdateNews(val offer: UpdateInfo, val ready: Boolean)
/**
* The one record of whether a newer app is on offer and where installing it
* has got to, shared by Home (which tells of it) and the About card (in
* Settings, and in the sheet Home's news opens). A singleton because each
* screen gets its own view model, and an install started from one must
* show on the other.
*
* Only the platform's answer retires an offer: declined, failed or refused,
* it stays on screen. A newer build is fetched ahead on Wi-Fi
* ([UpdateScheduler]), so the listener is usually told it is ready and
* Install is one tap. Home is told at most once a day for each build
* ([UpdateTold]).
*/
@Singleton
class AppUpdates @Inject constructor(
@ApplicationScope private val scope: CoroutineScope,
private val updates: UpdateRepository,
private val installer: ApkInstaller,
private val told: UpdateTold,
private val fetching: UpdateScheduler,
) {
private val mutable = MutableStateFlow(UpdateUiState())
val state: StateFlow<UpdateUiState> = mutable.asStateFlow()
private val telling = Channel<UpdateNews>(Channel.BUFFERED)
/** A newer build to tell of, once; a screen that shows it calls [tellHere] first. */
val news: Flow<UpdateNews> = telling.receiveAsFlow()
private var asking: Job? = null
private var tells = false
/**
* Asks the server which app it offers: when Home or Settings opens, on a
* pull, and every few minutes while one is up. A newer one being fetched
* or installed is never taken off the screen by a later answer.
*/
fun check(): Job {
asking?.takeIf { it.isActive }?.let { return it }
return scope.launch { refresh() }.also { asking = it }
}
private suspend fun refresh() {
mutable.update { it.copy(checking = true) }
val offer = updates.check()
mutable.update { now ->
val busy = now.stage != InstallStage.IDLE
when (offer) {
is Offer.Newer -> now.copy(offer = if (busy) now.offer else offer.info, unchecked = null)
Offer.Current -> now.copy(offer = if (busy) now.offer else null, unchecked = null)
is Offer.Problem -> now.copy(unchecked = offer.message)
}.copy(checking = false)
}
fetchOrTidy(offer)
lookForReady()
tellIfDue()
}
/** A newer build is fetched ahead; once none is offered, a package kept for one is only space. */
private suspend fun fetchOrTidy(offer: Offer) {
if (mutable.value.stage != InstallStage.IDLE) return
when (offer) {
is Offer.Newer -> fetching.fetchAhead()
Offer.Current -> updates.tidy()
is Offer.Problem -> Unit
}
}
/** Home is up: what is on offer now is told, if it is due. */
fun tellHere() {
tells = true
tellIfDue()
}
private fun tellIfDue() {
val now = mutable.value
val offer = now.offer
if (!tells || offer == null || now.stage != InstallStage.IDLE) return
val key = buildKey(offer.code, offer.version)
val at = System.currentTimeMillis()
if (dueToSay(key, told.build(), told.at(), at)) {
told.told(key, at)
telling.trySend(UpdateNews(offer, now.ready))
}
}
/** Whether the offer has been downloaded meanwhile, perhaps in the background. */
fun lookForReady() {
mutable.update { it.copy(ready = it.offer?.let(updates::ready) != null) }
}
fun mayInstall(): Boolean = installer.canInstall()
/** Sends the listener to allow installs; the update carries on when they come back. */
fun askToInstall() {
mutable.update { it.copy(problem = ALLOW) }
installer.requestInstallPermission()
}
/** Came back from allowing installs without allowing them. */
fun refused() {
mutable.update { it.copy(problem = REFUSED) }
}
/**
* Downloads the offer (or takes the one already here) and installs it.
* Runs in the app's scope, so leaving the screen does not stop it.
*/
fun fetchAndInstall() {
val offer = mutable.value.offer ?: return
if (mutable.value.stage != InstallStage.IDLE) return
mutable.update { it.copy(stage = InstallStage.DOWNLOADING, progress = 0f, problem = null) }
scope.launch {
val download = updates.download(offer) { done -> mutable.update { it.copy(progress = done) } }
when (download) {
is Download.Problem -> mutable.update {
it.copy(stage = InstallStage.IDLE, problem = download.message)
}
is Download.Ready -> {
mutable.update { it.copy(ready = true) }
install(download.apk)
}
}
}
}
private suspend fun install(apk: File) {
mutable.update { it.copy(stage = InstallStage.INSTALLING) }
// On a successful silent install this never returns: the process is replaced.
val outcome = installer.install(apk)
mutable.update { it.copy(stage = InstallStage.IDLE, problem = outcome.message()) }
}
private companion object {
const val ALLOW = "Allow Minstrel to install apps. The update carries on when you come back."
const val REFUSED =
"Minstrel is still not allowed to install apps, so nothing was installed. Tap Update to try again."
}
}
/**
* What coming back to the app does to an update the listener asked for and
* was sent to allow installs for.
*/
internal enum class AfterAsking {
/** Allowed now: carry on with the update they already asked for. */
FETCH,
/** Came back without allowing it. The offer stays; they are told. */
REFUSED,
/** Nothing asked, something already under way, or the offer not known yet. */
WAIT,
}
internal fun afterAsking(asked: Boolean, mayInstall: Boolean, offered: Boolean, busy: Boolean): AfterAsking =
when {
!asked || busy || !offered -> AfterAsking.WAIT
mayInstall -> AfterAsking.FETCH
else -> AfterAsking.REFUSED
}
@@ -26,37 +26,22 @@ sealed interface InstallOutcome {
}
/**
* Where an install has got to, for the two surfaces that show it: the shell's
* [UpdateBanner][com.fabledsword.minstrel.update.ui.UpdateBanner] and the
* Settings About card.
* Where an install has got to, for the card that shows it (in Settings and
* in the sheet Home's update news opens).
*
* DOWNLOADING and INSTALLING are deliberately distinct: the install awaits
* the platform's verdict and can sit on a confirm dialog indefinitely, so
* one combined state would leave the UI claiming "Downloading…" through it.
* A failure is not a stage: it returns to IDLE with the reason beside it,
* and the offer stays so the listener can try again.
*/
enum class InstallStage { IDLE, DOWNLOADING, INSTALLING, ERROR }
/** True while an install is underway and a second tap should do nothing. */
fun InstallStage.isBusy(): Boolean =
this == InstallStage.DOWNLOADING || this == InstallStage.INSTALLING
/**
* The stage an outcome lands the UI in. A cancelled install returns to IDLE
* rather than ERROR — the user chose it, so presenting it as a failure would
* be a lie with a red tint.
*/
fun InstallOutcome.stage(): InstallStage = when (this) {
InstallOutcome.Installed, InstallOutcome.Cancelled -> InstallStage.IDLE
is InstallOutcome.Failed -> InstallStage.ERROR
}
enum class InstallStage { IDLE, DOWNLOADING, INSTALLING }
/**
* User-facing copy for an outcome; null when there is nothing worth saying.
*
* Lives beside the outcome rather than in either UI package because two
* separate screens surface the same verdicts and must not drift — the same
* reasoning that puts [ErrorCopy][com.fabledsword.minstrel.api.ErrorCopy]
* outside the UI layer.
* Lives beside the outcome rather than in the UI package, the same reasoning
* that puts [ErrorCopy][com.fabledsword.minstrel.api.ErrorCopy] outside the
* UI layer.
*/
fun InstallOutcome.message(): String? = when (this) {
InstallOutcome.Installed -> null
@@ -1,78 +0,0 @@
package com.fabledsword.minstrel.update.data
import com.fabledsword.minstrel.BuildConfig
import com.fabledsword.minstrel.di.ApplicationScope
import com.fabledsword.minstrel.models.UpdateInfo
import kotlinx.coroutines.CoroutineScope
import kotlinx.coroutines.delay
import kotlinx.coroutines.flow.MutableStateFlow
import kotlinx.coroutines.flow.SharingStarted
import kotlinx.coroutines.flow.StateFlow
import kotlinx.coroutines.flow.combine
import kotlinx.coroutines.flow.stateIn
import kotlinx.coroutines.launch
import javax.inject.Inject
import javax.inject.Singleton
private const val POLL_INTERVAL_MS = 24 * 60 * 60 * 1000L
/**
* Drives the shell's soft "update available" banner. Polls
* `/api/client/version` at launch + every 24h and, when the bundled
* APK outranks this build — by ordering key where the server reports one,
* by name otherwise — exposes its [UpdateInfo] so
* [com.fabledsword.minstrel.update.ui.UpdateBanner] can nudge an
* install.
*
* Dismissals are kept in memory and keyed by version, so dismissing
* one release's banner still lets a later release re-surface — and a
* restart re-shows it, which is acceptable nudging. Server 404 /
* network errors stay silent. Constructed at
* launch via the construct-the-singleton trick in `MinstrelApplication`.
*
* A debug build never polls: it is signed with a local debug key, so the
* server's release-signed APK could never install over it.
*/
@Singleton
class UpdateBannerController @Inject constructor(
@ApplicationScope private val scope: CoroutineScope,
private val repository: UpdateRepository,
) {
private val latest = MutableStateFlow<UpdateInfo?>(null)
private val dismissed = MutableStateFlow<Set<String>>(emptySet())
/** The available update, or null when none / dismissed / unreachable. */
val available: StateFlow<UpdateInfo?> =
combine(latest, dismissed) { info, seen ->
info?.takeIf { it.version !in seen }
}.stateIn(scope, SharingStarted.Eagerly, null)
init {
if (!BuildConfig.DEBUG) startPolling()
}
private fun startPolling() {
scope.launch {
while (true) {
runOnce()
delay(POLL_INTERVAL_MS)
}
}
}
fun dismiss(version: String) {
dismissed.value = dismissed.value + version
}
private suspend fun runOnce() {
val info = runCatching { repository.getLatest() }.getOrNull() ?: return
latest.value = info.takeIf {
isUpdateAvailable(
serverCode = it.code,
serverName = it.version,
installedCode = BuildConfig.VERSION_CODE.toLong(),
installedName = BuildConfig.VERSION_NAME,
)
}
}
}
@@ -0,0 +1,84 @@
package com.fabledsword.minstrel.update.data
import android.content.Context
import androidx.hilt.work.HiltWorker
import androidx.work.Constraints
import androidx.work.CoroutineWorker
import androidx.work.ExistingWorkPolicy
import androidx.work.NetworkType
import androidx.work.OneTimeWorkRequestBuilder
import androidx.work.WorkManager
import androidx.work.WorkerParameters
import dagger.assisted.Assisted
import dagger.assisted.AssistedInject
import dagger.hilt.android.qualifiers.ApplicationContext
import javax.inject.Inject
import javax.inject.Singleton
/**
* Brings the newer app down before the listener is asked to install it, so
* Install is one tap rather than a wait. It never installs and shows
* nothing: whether to update stays the listener's.
*
* It asks the server again rather than being told what to fetch. WorkManager
* may run it hours after it was asked for, when Wi-Fi comes, and by then the
* server may offer another build.
*/
@HiltWorker
class UpdateFetchWorker @AssistedInject constructor(
@Assisted context: Context,
@Assisted params: WorkerParameters,
private val updates: UpdateRepository,
) : CoroutineWorker(context, params) {
override suspend fun doWork(): Result = when (val offer = updates.check()) {
// A server out of reach is a phone away from home; the next check asks for this again.
is Offer.Problem -> Result.success()
Offer.Current -> {
updates.tidy()
Result.success()
}
is Offer.Newer -> when (updates.download(offer.info) {}) {
is Download.Ready -> Result.success()
// A dropped connection is worth another go after WorkManager's
// backoff, which also outlasts the server's one-download-a-minute
// limit; a few, not forever.
is Download.Problem -> if (runAttemptCount < TRIES) Result.retry() else Result.success()
}
}
private companion object {
const val TRIES = 3
}
}
/**
* Asks the platform to fetch the newer app when it is free to: on an
* unmetered network, with the battery and storage not low. Nothing turns
* this onto mobile data, because the listener's Update still downloads at
* once over whatever network there is; this only saves the wait.
*
* Kept rather than replaced, so a check every few minutes never restarts a
* download already under way.
*/
@Singleton
class UpdateScheduler @Inject constructor(@ApplicationContext private val context: Context) {
fun fetchAhead() {
WorkManager.getInstance(context).enqueueUniqueWork(
FETCHING,
ExistingWorkPolicy.KEEP,
OneTimeWorkRequestBuilder<UpdateFetchWorker>()
.setConstraints(
Constraints.Builder()
.setRequiredNetworkType(NetworkType.UNMETERED)
.setRequiresBatteryNotLow(true)
.setRequiresStorageNotLow(true)
.build(),
)
.build(),
)
}
private companion object {
const val FETCHING = "update-fetch"
}
}
@@ -0,0 +1,55 @@
package com.fabledsword.minstrel.update.data
import android.content.Context
import com.fabledsword.minstrel.BuildConfig
import com.fabledsword.minstrel.update.api.ClientVersionApi
import dagger.Module
import dagger.Provides
import dagger.hilt.InstallIn
import dagger.hilt.android.qualifiers.ApplicationContext
import dagger.hilt.components.SingletonComponent
import kotlinx.coroutines.Dispatchers
import kotlinx.coroutines.withContext
import okhttp3.OkHttpClient
import okhttp3.Request
import retrofit2.Retrofit
import java.io.File
import java.io.IOException
import javax.inject.Singleton
@Module
@InstallIn(SingletonComponent::class)
object UpdateModule {
/**
* The download goes through the shared [OkHttpClient] so it inherits the
* auth cookie and the BaseUrlInterceptor host rewrite (apkUrl is
* server-relative, e.g. `/api/client/apk`). A refusal, such as the
* server's one-download-a-minute limit, reads as an IOException, which
* the repository reports and the fetch worker retries after a backoff.
*/
@Provides
@Singleton
fun provideUpdateRepository(
@ApplicationContext context: Context,
retrofit: Retrofit,
okHttpClient: OkHttpClient,
): UpdateRepository = UpdateRepository(
api = retrofit.create(ClientVersionApi::class.java),
apk = ApkSource { apkUrl ->
withContext(Dispatchers.IO) {
val request = Request.Builder().url("http://placeholder.invalid$apkUrl").build()
val response = okHttpClient.newCall(request).execute()
if (!response.isSuccessful) {
response.close()
throw IOException("Download failed: HTTP ${response.code}")
}
response.body ?: throw IOException("Empty download body")
}
},
dir = File(context.cacheDir, "update"),
installedCode = BuildConfig.VERSION_CODE.toLong(),
installedName = BuildConfig.VERSION_NAME,
offers = !BuildConfig.DEBUG,
legacyApk = File(context.cacheDir, "minstrel-update.apk"),
)
}
@@ -1,21 +1,158 @@
package com.fabledsword.minstrel.update.data
import com.fabledsword.minstrel.api.ErrorCopy
import com.fabledsword.minstrel.models.UpdateInfo
import com.fabledsword.minstrel.models.wire.UpdateInfoWire
import com.fabledsword.minstrel.update.api.ClientVersionApi
import retrofit2.Retrofit
import javax.inject.Inject
import javax.inject.Singleton
import kotlinx.coroutines.Dispatchers
import kotlinx.coroutines.sync.Mutex
import kotlinx.coroutines.sync.withLock
import kotlinx.coroutines.withContext
import kotlinx.serialization.SerializationException
import okhttp3.ResponseBody
import retrofit2.HttpException
import java.io.File
import java.io.IOException
import java.io.InputStream
import java.io.OutputStream
import java.net.HttpURLConnection.HTTP_NOT_FOUND
import java.security.MessageDigest
import kotlin.coroutines.cancellation.CancellationException
/** What asking the server for its app came to. */
sealed interface Offer {
/** The server's app is another build than this one, and installs over it. */
data class Newer(val info: UpdateInfo) : Offer
/** This is the server's app, or the same app built again, or the server has none to offer. */
data object Current : Offer
/** The server could not be asked. [message] says why, for the About card. */
data class Problem(val message: String) : Offer
}
/** What a download came to. */
sealed interface Download {
data class Ready(val apk: File) : Download
data class Problem(val message: String) : Download
}
/** Opens the server's APK for reading. The caller closes the body. */
fun interface ApkSource {
suspend fun open(apkUrl: String): ResponseBody
}
/**
* Thin facade over `/api/client/version`, used by the About card's
* Check for updates button and by [UpdateBannerController].
* Whether the server offers a newer app than this one, and fetching it.
*
* One way a package arrives, whether [UpdateFetchWorker] brought it down
* ahead on Wi-Fi or the listener pressed Update: [download]. The two take
* turns, and each looks first for what the other already brought, so
* nothing is downloaded twice. A package only gets its build's name once
* it checked out against the size and digest the server gave, so a file at
* that name is that build, whole ([ready]).
*
* A debug build is never offered one ([offers] false): it is signed with the
* debug key, so the platform would refuse the release build over it anyway.
*/
@Singleton
class UpdateRepository @Inject constructor(retrofit: Retrofit) {
private val api: ClientVersionApi = retrofit.create(ClientVersionApi::class.java)
class UpdateRepository(
private val api: ClientVersionApi,
private val apk: ApkSource,
private val dir: File,
private val installedCode: Long,
private val installedName: String,
private val offers: Boolean,
// A download left by older builds of the app, which nothing else removes.
private val legacyApk: File? = null,
) {
private val turns = Mutex()
suspend fun getLatest(): UpdateInfo = api.getVersion().toDomain()
suspend fun check(): Offer {
if (!offers) return Offer.Current
return try {
val info = api.getVersion().toDomain()
if (isUpdateAvailable(info.code, info.version, installedCode, installedName)) {
Offer.Newer(info)
} else {
Offer.Current
}
} catch (e: CancellationException) {
throw e
} catch (e: HttpException) {
// 404 is the server saying it bundles no app: nothing to offer.
if (e.code() == HTTP_NOT_FOUND) Offer.Current else Offer.Problem(ErrorCopy.fromThrowable(e))
} catch (e: IOException) {
Offer.Problem(ErrorCopy.fromThrowable(e))
} catch (e: SerializationException) {
Offer.Problem(ErrorCopy.fromThrowable(e))
}
}
/** The package for [info], when it is already here and whole. */
fun ready(info: UpdateInfo): File? =
waiting(info).takeIf { it.isFile && it.length() == info.sizeBytes }
/**
* The server's app, checked against the size and digest the server gave,
* so what is handed to the installer is the build that was offered, all
* of it. [progress] is called with 0 to 1 as it arrives.
*/
suspend fun download(info: UpdateInfo, progress: (Float) -> Unit): Download =
turns.withLock { ready(info)?.let { Download.Ready(it) } ?: fetch(info, progress) }
/** Drops every package kept here: none is on offer any more. */
suspend fun tidy() {
turns.withLock {
withContext(Dispatchers.IO) {
dir.listFiles()?.forEach { it.delete() }
legacyApk?.delete()
}
}
}
/**
* Writes to a part file and gives it the build's name only once it
* checks out, so a download cut off partway is never taken for one that
* finished. Any package for another build goes then.
*/
private suspend fun fetch(info: UpdateInfo, progress: (Float) -> Unit): Download {
val target = waiting(info)
val part = File(dir, target.name + PART)
return try {
val digest = withContext(Dispatchers.IO) {
save(apk.open(info.apkUrl), part, info.sizeBytes, progress)
}
withContext(Dispatchers.IO) { settle(info, part, target, digest) }
} catch (e: CancellationException) {
part.delete()
throw e
} catch (e: IOException) {
part.delete()
Download.Problem("Couldn't download the update: ${ErrorCopy.fromThrowable(e)}")
}
}
private fun settle(info: UpdateInfo, part: File, target: File, digest: String): Download {
// A server that predates the digest leaves the size as the only check.
val whole = part.length() == info.sizeBytes &&
(info.sha256.isEmpty() || digest.equals(info.sha256, ignoreCase = true))
return if (whole && part.renameTo(target)) {
dir.listFiles()?.filter { it != target }?.forEach { it.delete() }
Download.Ready(target)
} else {
part.delete()
Download.Problem("The download did not match the app the server offered. Try again.")
}
}
/** Where the package for [info] waits: named for its build. */
private fun waiting(info: UpdateInfo): File =
File(dir, "minstrel-${info.code ?: info.version.replace(UNSAFE, "_")}.apk")
private companion object {
const val PART = ".part"
val UNSAFE = Regex("[^A-Za-z0-9._-]")
}
}
private fun UpdateInfoWire.toDomain(): UpdateInfo = UpdateInfo(
@@ -24,20 +161,55 @@ private fun UpdateInfoWire.toDomain(): UpdateInfo = UpdateInfo(
channel = channel,
apkUrl = apkUrl,
sizeBytes = sizeBytes,
sha256 = sha256,
)
/** Writes [body] to [file], reporting progress against [size], and returns the SHA-256 written, in hex. */
internal fun save(body: ResponseBody, file: File, size: Long, progress: (Float) -> Unit): String {
file.parentFile?.mkdirs()
val digest = MessageDigest.getInstance("SHA-256")
// Closing the stream closes the body.
body.byteStream().use { input ->
file.outputStream().use { output -> copy(input, output, digest) { written -> report(written, size, progress) } }
}
return digest.digest().joinToString("") { "%02x".format(it) }
}
private fun copy(input: InputStream, output: OutputStream, digest: MessageDigest, written: (Long) -> Unit) {
val buffer = ByteArray(BUFFER_BYTES)
var total = 0L
var read = input.read(buffer)
while (read >= 0) {
output.write(buffer, 0, read)
digest.update(buffer, 0, read)
total += read
written(total)
read = input.read(buffer)
}
}
private fun report(written: Long, size: Long, progress: (Float) -> Unit) {
if (size > 0) progress((written.toFloat() / size).coerceAtMost(1f))
}
private const val BUFFER_BYTES = 64 * 1024
/**
* True when [server] should be offered over the installed build.
* True when [serverName] / [serverCode] should be offered over the installed
* build. Two questions, each asked of the value that answers it.
*
* **Decide on the ordering key whenever the server sends one.** That is the
* same value Android's package installer compares, so an offer made this way
* implies an install the platform will actually accept. Comparing NAMES
* while the platform installs by `versionCode` keeps nothing consistent
* between the two orderings: it can offer a build Android then refuses as
* a downgrade, or stay quiet about one it would take.
* **May it be installed over this one** is the ordering key's, whenever the
* server sends one: it is the same value Android's package installer
* compares, so an offer made on it is an install the platform will take.
*
* Name comparison survives only as the fallback for a server that predates
* the field. A null code means "this server cannot tell me" — never "zero" —
* **Is it another app at all** is the name's, compared for equality only.
* Every server image builds the app again at a later minute, so the code
* rises with a CI re-run or a change outside the shipped set while the app
* itself does not; asking only the first question offered the installed app
* to itself after each one.
*
* Name ordering survives only as the fallback for a server that predates the
* code. A null code means "this server cannot tell me" — never "zero" —
* because treating absent as zero would rank every such server as infinitely
* old and offer its build to everyone, forever.
*/
@@ -48,7 +220,7 @@ fun isUpdateAvailable(
installedName: String,
): Boolean =
if (serverCode != null) {
serverCode > installedCode
serverCode > installedCode && serverName != installedName
} else {
isVersionNewer(serverName, installedName)
}
@@ -0,0 +1,58 @@
package com.fabledsword.minstrel.update.data
import android.content.Context
import dagger.hilt.android.qualifiers.ApplicationContext
import javax.inject.Inject
import javax.inject.Singleton
/**
* The build Home last told the listener of, and when: on disk, so an app
* Android reopened does not say the same news again.
*/
@Singleton
class UpdateTold @Inject constructor(@ApplicationContext context: Context) {
private val prefs = context.getSharedPreferences(PREFS_NAME, Context.MODE_PRIVATE)
fun build(): String = prefs.getString(BUILD, "").orEmpty()
fun at(): Long = prefs.getLong(AT, 0L)
fun told(build: String, at: Long) {
prefs.edit().putString(BUILD, build).putLong(AT, at).apply()
}
private companion object {
const val PREFS_NAME = "update_told"
const val BUILD = "build"
const val AT = "at"
}
}
/**
* The key a build is told under: its code, or its name from a server that
* sends no code.
*/
internal fun buildKey(code: Long?, version: String): String = code?.toString() ?: version
/**
* Whether a build on offer is said again: a build not yet told of is news at
* once; the same one again a day later, until it is installed. A clock that
* went backwards says it rather than hiding an update for a day.
*/
internal fun dueToSay(build: String, toldBuild: String, toldAt: Long, now: Long): Boolean {
val since = now - toldAt
return build != toldBuild || since < 0 || since >= SAY_AGAIN_AFTER_MS
}
/** What Home's snackbar says of an offer; its button opens the sheet. */
internal fun toldWords(version: String, ready: Boolean): String =
if (ready) {
"Version $version of the app is downloaded and ready."
} else {
"Version $version of the app is on your server."
}
/** The snackbar's button: Install when there is nothing left to download. */
internal fun toldAction(ready: Boolean): String = if (ready) "Install" else "Update"
internal const val SAY_AGAIN_AFTER_MS = 24L * 60 * 60 * 1000
@@ -0,0 +1,89 @@
package com.fabledsword.minstrel.update.ui
import androidx.compose.foundation.layout.fillMaxWidth
import androidx.compose.material3.Button
import androidx.compose.material3.LinearProgressIndicator
import androidx.compose.material3.MaterialTheme
import androidx.compose.material3.Text
import androidx.compose.runtime.Composable
import androidx.compose.runtime.getValue
import androidx.compose.ui.Modifier
import androidx.lifecycle.compose.LifecycleResumeEffect
import androidx.lifecycle.compose.collectAsStateWithLifecycle
import com.fabledsword.minstrel.BuildConfig
import com.fabledsword.minstrel.models.UpdateInfo
import com.fabledsword.minstrel.settings.ui.formatBytes
import com.fabledsword.minstrel.theme.Muted
import com.fabledsword.minstrel.theme.RowSpinner
import com.fabledsword.minstrel.theme.RowValue
import com.fabledsword.minstrel.theme.Section
import com.fabledsword.minstrel.theme.SettingRow
import com.fabledsword.minstrel.update.data.InstallStage
import com.fabledsword.minstrel.update.data.UpdateUiState
/**
* The app's own version, the newer one when the server has it, and a check
* that is always there. Settings shows it as its About card, and the sheet
* Home's update news opens holds the very same card, so there is one place
* the app installs from. Back from allowing installs, the update the
* listener asked for carries on.
*/
@Composable
fun AppVersionCard(model: UpdateViewModel, modifier: Modifier = Modifier) {
LifecycleResumeEffect(model) {
model.resumed()
onPauseOrDispose {}
}
val state by model.state.collectAsStateWithLifecycle()
Section("About", modifier) {
SettingRow(title = "Version") {
RowValue("${BuildConfig.VERSION_NAME} (${BuildConfig.VERSION_CODE})")
}
val offer = state.offer
if (offer != null) {
Offered(offer, state, model::update)
} else {
val canCheck = !BuildConfig.DEBUG && !state.checking && state.stage == InstallStage.IDLE
SettingRow(
title = "Check for updates",
summary = versionWords(state, selfUpdates = !BuildConfig.DEBUG),
enabled = canCheck || state.checking,
onClick = if (canCheck) model::check else null,
) { if (state.checking) RowSpinner() }
}
state.problem?.let { Muted(it, tone = MaterialTheme.colorScheme.error) }
}
}
/** The newer build, and the one button that fetches and installs it. */
@Composable
private fun Offered(offer: UpdateInfo, state: UpdateUiState, update: () -> Unit) {
SettingRow(
title = if (state.ready) "Ready to install" else "New version on your server",
summary = describe(offer),
) {
when (state.stage) {
InstallStage.IDLE -> Button(onClick = update) { Text(if (state.ready) "Install" else "Update") }
InstallStage.DOWNLOADING -> RowSpinner()
InstallStage.INSTALLING -> RowValue("Installing…")
}
}
if (state.stage == InstallStage.DOWNLOADING) {
LinearProgressIndicator(progress = { state.progress }, modifier = Modifier.fillMaxWidth())
}
}
/** Whether this app is the newest, as far as the server has said. */
internal fun versionWords(state: UpdateUiState, selfUpdates: Boolean = true): String = when {
// Its own package and the debug key: the server's app would install beside it, never over it.
!selfUpdates -> "Debug build: updates install from Android Studio."
state.checking -> "Checking for a newer version…"
state.unchecked != null -> "Couldn't check for a newer version: ${state.unchecked}"
else -> "This is the newest version."
}
/** The name, the channel when it is the test line, and the size. */
internal fun describe(offer: UpdateInfo): String {
val channel = if (offer.channel == "dev") " (test build)" else ""
return "${offer.version}$channel · ${formatBytes(offer.sizeBytes)}"
}
@@ -1,149 +0,0 @@
package com.fabledsword.minstrel.update.ui
import androidx.compose.animation.AnimatedVisibility
import androidx.compose.animation.expandVertically
import androidx.compose.animation.fadeIn
import androidx.compose.animation.fadeOut
import androidx.compose.animation.shrinkVertically
import androidx.compose.foundation.background
import androidx.compose.foundation.layout.Arrangement
import androidx.compose.foundation.layout.Column
import androidx.compose.foundation.layout.Row
import androidx.compose.foundation.layout.fillMaxWidth
import androidx.compose.foundation.layout.padding
import androidx.compose.material3.Icon
import androidx.compose.material3.IconButton
import androidx.compose.material3.LinearProgressIndicator
import androidx.compose.material3.MaterialTheme
import androidx.compose.material3.Text
import androidx.compose.material3.TextButton
import androidx.compose.runtime.Composable
import androidx.compose.runtime.getValue
import androidx.compose.ui.Alignment
import androidx.compose.ui.Modifier
import androidx.compose.ui.unit.dp
import androidx.hilt.navigation.compose.hiltViewModel
import androidx.lifecycle.compose.collectAsStateWithLifecycle
import com.composables.icons.lucide.Download
import com.composables.icons.lucide.Lucide
import com.composables.icons.lucide.X
import com.fabledsword.minstrel.models.UpdateInfo
import com.fabledsword.minstrel.update.data.InstallStage
import com.fabledsword.minstrel.update.data.isBusy
/**
* Shell-level soft banner that nudges an available update. Renders
* nothing until [UpdateBannerController] surfaces a newer bundled APK
* (and the user hasn't dismissed that version): a download glyph,
* "Update Minstrel · {version} available", an Install button
* (download → system installer), and a dismiss X. Uses an understated
* surface tone, not the alarming error color reserved for
* `VersionTooOldBanner`.
*
* [ApkInstaller][com.fabledsword.minstrel.update.data.ApkInstaller]
* exposes no byte progress, so the downloading state is an
* indeterminate bar.
*/
@Composable
fun UpdateBanner(viewModel: UpdateBannerViewModel = hiltViewModel()) {
val state by viewModel.uiState.collectAsStateWithLifecycle()
val info = state.info
AnimatedVisibility(
visible = info != null,
enter = expandVertically() + fadeIn(),
exit = shrinkVertically() + fadeOut(),
) {
// info can flip to null during the exit animation; hold the last
// non-null so the collapsing banner keeps its content.
val shown = info ?: return@AnimatedVisibility
BannerBody(
info = shown,
stage = state.stage,
message = state.message,
onInstall = { viewModel.install(shown) },
onDismiss = { viewModel.dismiss(shown.version) },
)
}
}
@Composable
private fun BannerBody(
info: UpdateInfo,
stage: InstallStage,
message: String?,
onInstall: () -> Unit,
onDismiss: () -> Unit,
) {
Column(
modifier = Modifier
.fillMaxWidth()
.background(MaterialTheme.colorScheme.surfaceVariant)
.padding(start = 16.dp, top = 8.dp, end = 4.dp, bottom = 8.dp),
) {
BannerRow(info = info, stage = stage, onInstall = onInstall, onDismiss = onDismiss)
if (stage.isBusy()) {
LinearProgressIndicator(
modifier = Modifier
.fillMaxWidth()
.padding(top = 4.dp),
color = MaterialTheme.colorScheme.primary,
)
}
if (message != null) {
Text(
text = message,
style = MaterialTheme.typography.bodySmall,
color = MaterialTheme.colorScheme.onSurfaceVariant,
modifier = Modifier.padding(top = 4.dp),
)
}
}
}
@Composable
private fun BannerRow(
info: UpdateInfo,
stage: InstallStage,
onInstall: () -> Unit,
onDismiss: () -> Unit,
) {
Row(
verticalAlignment = Alignment.CenterVertically,
horizontalArrangement = Arrangement.spacedBy(8.dp),
) {
Icon(
imageVector = Lucide.Download,
contentDescription = null,
tint = MaterialTheme.colorScheme.onSurfaceVariant,
)
Text(
text = if (stage == InstallStage.ERROR) {
"Update failed"
} else {
"Update Minstrel · ${info.version} available"
},
style = MaterialTheme.typography.bodySmall,
color = MaterialTheme.colorScheme.onSurfaceVariant,
modifier = Modifier.weight(1f),
)
TextButton(onClick = onInstall, enabled = !stage.isBusy()) {
// Downloading and installing are separate words because they're now
// separate waits — the install half suspends on the platform, which
// may be sitting on a confirm dialog.
Text(
when (stage) {
InstallStage.DOWNLOADING -> "Downloading…"
InstallStage.INSTALLING -> "Installing…"
else -> "Install"
},
)
}
IconButton(onClick = onDismiss) {
Icon(
imageVector = Lucide.X,
contentDescription = "Dismiss",
tint = MaterialTheme.colorScheme.onSurfaceVariant,
)
}
}
}
@@ -1,92 +0,0 @@
package com.fabledsword.minstrel.update.ui
import androidx.lifecycle.ViewModel
import androidx.lifecycle.viewModelScope
import com.fabledsword.minstrel.api.ErrorCopy
import com.fabledsword.minstrel.models.UpdateInfo
import com.fabledsword.minstrel.update.data.ApkInstaller
import com.fabledsword.minstrel.update.data.InstallStage
import com.fabledsword.minstrel.update.data.UpdateBannerController
import com.fabledsword.minstrel.update.data.isBusy
import com.fabledsword.minstrel.update.data.message
import com.fabledsword.minstrel.update.data.stage
import dagger.hilt.android.lifecycle.HiltViewModel
import kotlinx.coroutines.flow.MutableStateFlow
import kotlinx.coroutines.flow.SharingStarted
import kotlinx.coroutines.flow.StateFlow
import kotlinx.coroutines.flow.combine
import kotlinx.coroutines.flow.stateIn
import kotlinx.coroutines.launch
import java.io.File
import javax.inject.Inject
private const val SHARE_STOP_TIMEOUT_MS = 5_000L
data class UpdateBannerUiState(
val info: UpdateInfo? = null,
val stage: InstallStage = InstallStage.IDLE,
val message: String? = null,
)
/**
* Thin VM over [UpdateBannerController]. Surfaces the available update
* and runs the download → install handoff via [ApkInstaller], mirroring
* the About card's flow (route to "install unknown apps" settings first
* when the permission is missing).
*/
@HiltViewModel
class UpdateBannerViewModel @Inject constructor(
private val controller: UpdateBannerController,
private val installer: ApkInstaller,
) : ViewModel() {
private val installState = MutableStateFlow(InstallSnapshot(InstallStage.IDLE, null))
val uiState: StateFlow<UpdateBannerUiState> =
combine(controller.available, installState) { info, install ->
UpdateBannerUiState(info = info, stage = install.stage, message = install.message)
}.stateIn(
scope = viewModelScope,
started = SharingStarted.WhileSubscribed(SHARE_STOP_TIMEOUT_MS),
initialValue = UpdateBannerUiState(),
)
fun dismiss(version: String) = controller.dismiss(version)
fun install(info: UpdateInfo) {
if (installState.value.stage.isBusy()) return
if (!installer.canInstall()) {
installer.requestInstallPermission()
installState.value = InstallSnapshot(
InstallStage.IDLE,
"Allow installs for Minstrel, then tap Install again.",
)
return
}
viewModelScope.launch {
installState.value = InstallSnapshot(InstallStage.DOWNLOADING, null)
val apk = download(info.apkUrl)
if (apk != null) {
// Await the platform's verdict rather than firing an intent and
// assuming it worked. On a silent install this suspends until
// the process is replaced, so the line below is only reached
// when the install did NOT simply succeed.
installState.value = InstallSnapshot(InstallStage.INSTALLING, null)
val outcome = installer.install(apk)
installState.value = InstallSnapshot(outcome.stage(), outcome.message())
}
}
}
private suspend fun download(apkUrl: String): File? =
runCatching { installer.downloadApk(apkUrl) }
.onFailure { e ->
installState.value = InstallSnapshot(
InstallStage.ERROR,
"Couldn't download update: ${ErrorCopy.fromThrowable(e)}",
)
}
.getOrNull()
}
private data class InstallSnapshot(val stage: InstallStage, val message: String?)
@@ -0,0 +1,88 @@
package com.fabledsword.minstrel.update.ui
import androidx.compose.foundation.layout.BoxScope
import androidx.compose.foundation.layout.fillMaxWidth
import androidx.compose.foundation.layout.navigationBarsPadding
import androidx.compose.foundation.layout.padding
import androidx.compose.material3.ExperimentalMaterial3Api
import androidx.compose.material3.ModalBottomSheet
import androidx.compose.material3.SnackbarDuration
import androidx.compose.material3.SnackbarHost
import androidx.compose.material3.SnackbarHostState
import androidx.compose.material3.SnackbarResult
import androidx.compose.runtime.Composable
import androidx.compose.runtime.LaunchedEffect
import androidx.compose.runtime.getValue
import androidx.compose.runtime.mutableStateOf
import androidx.compose.runtime.remember
import androidx.compose.runtime.saveable.rememberSaveable
import androidx.compose.runtime.setValue
import androidx.compose.ui.Alignment
import androidx.compose.ui.Modifier
import androidx.compose.ui.unit.dp
import androidx.lifecycle.Lifecycle
import androidx.lifecycle.compose.LocalLifecycleOwner
import androidx.lifecycle.repeatOnLifecycle
import com.fabledsword.minstrel.update.data.toldAction
import com.fabledsword.minstrel.update.data.toldWords
import kotlinx.coroutines.delay
/** How often a screen on show asks the server about a newer app by itself. */
private const val CHECK_EVERY_MS = 5 * 60 * 1000L
/**
* A newer app, told on Home (after Librarian's and Gallery's update tray): a
* snackbar at the foot of the screen, never a dialog, and its button opens a
* sheet holding the very card Settings shows, so there is one place the app
* installs from. Each build is told at most once a day. The sheet stays open
* through a trip to allow installs, which is where the card carries on from.
*/
@Composable
fun BoxScope.UpdateNotices(model: UpdateViewModel) {
val notices = remember { SnackbarHostState() }
var updating by rememberSaveable { mutableStateOf(false) }
CheckWhileShown(model)
LaunchedEffect(model) {
model.tellHere()
model.news.collect { news ->
val tapped = notices.showSnackbar(
message = toldWords(news.offer.version, news.ready),
actionLabel = toldAction(news.ready),
withDismissAction = true,
duration = SnackbarDuration.Long,
)
if (tapped == SnackbarResult.ActionPerformed) updating = true
}
}
SnackbarHost(notices, Modifier.align(Alignment.BottomCenter).padding(12.dp))
if (updating) UpdateSheet(model) { updating = false }
}
@OptIn(ExperimentalMaterial3Api::class)
@Composable
private fun UpdateSheet(model: UpdateViewModel, dismiss: () -> Unit) {
ModalBottomSheet(onDismissRequest = dismiss) {
AppVersionCard(
model,
Modifier.navigationBarsPadding().padding(start = 16.dp, end = 16.dp, bottom = 16.dp),
)
}
}
/**
* Asks the server about a newer app when the screen opens, as soon as the
* app comes back to the front, and every [CHECK_EVERY_MS] while it is on
* screen. A screen in the background never asks.
*/
@Composable
fun CheckWhileShown(model: UpdateViewModel) {
val lifecycle = LocalLifecycleOwner.current.lifecycle
LaunchedEffect(model, lifecycle) {
lifecycle.repeatOnLifecycle(Lifecycle.State.RESUMED) {
while (true) {
model.check()
delay(CHECK_EVERY_MS)
}
}
}
}
@@ -0,0 +1,86 @@
package com.fabledsword.minstrel.update.ui
import androidx.lifecycle.SavedStateHandle
import androidx.lifecycle.ViewModel
import androidx.lifecycle.viewModelScope
import com.fabledsword.minstrel.update.data.AfterAsking
import com.fabledsword.minstrel.update.data.AppUpdates
import com.fabledsword.minstrel.update.data.InstallStage
import com.fabledsword.minstrel.update.data.UpdateNews
import com.fabledsword.minstrel.update.data.UpdateUiState
import com.fabledsword.minstrel.update.data.afterAsking
import dagger.hilt.android.lifecycle.HiltViewModel
import kotlinx.coroutines.flow.Flow
import kotlinx.coroutines.flow.StateFlow
import kotlinx.coroutines.launch
import javax.inject.Inject
/**
* A screen's handle on [AppUpdates]. What it adds is the one thing that
* belongs to the screen: that the listener pressed Update and was sent to
* allow installs. That is kept in [saved], so it outlives the process
* Android may restart from the settings page, and the update carries on
* when they come back ([resumed]) instead of asking them to tap again.
*/
@HiltViewModel
class UpdateViewModel @Inject constructor(
private val updates: AppUpdates,
private val saved: SavedStateHandle,
) : ViewModel() {
val state: StateFlow<UpdateUiState> = updates.state
val news: Flow<UpdateNews> = updates.news
/** Asks the server again; once the answer is in, an update asked for before a restart carries on. */
fun check() {
viewModelScope.launch {
updates.check().join()
carryOn()
}
}
fun tellHere() = updates.tellHere()
/** The app is in front again, perhaps back from allowing installs. */
fun resumed() {
updates.lookForReady()
carryOn()
}
fun update() {
val now = state.value
when {
now.offer == null || now.stage != InstallStage.IDLE -> Unit
!updates.mayInstall() -> {
saved[ASKED] = true
updates.askToInstall()
}
else -> updates.fetchAndInstall()
}
}
private fun carryOn() {
val now = state.value
val next = afterAsking(
asked = saved.get<Boolean>(ASKED) == true,
mayInstall = updates.mayInstall(),
offered = now.offer != null,
busy = now.stage != InstallStage.IDLE,
)
when (next) {
AfterAsking.WAIT -> Unit
AfterAsking.REFUSED -> {
saved[ASKED] = false
updates.refused()
}
AfterAsking.FETCH -> {
saved[ASKED] = false
updates.fetchAndInstall()
}
}
}
private companion object {
/** The listener asked for the update and was sent to allow installs. */
const val ASKED = "update.asked_to_allow"
}
}
@@ -2,23 +2,32 @@ package com.fabledsword.minstrel.api
import com.fabledsword.minstrel.auth.AuthStore
import com.fabledsword.minstrel.auth.FakeSessionVault
import com.fabledsword.minstrel.auth.SessionVault
import com.fabledsword.minstrel.cache.db.dao.AuthSessionDao
import com.fabledsword.minstrel.cache.db.entities.AuthSessionEntity
import io.mockk.coEvery
import io.mockk.every
import io.mockk.mockk
import kotlinx.coroutines.CoroutineScope
import kotlinx.coroutines.Dispatchers
import kotlinx.coroutines.ExperimentalCoroutinesApi
import kotlinx.coroutines.SupervisorJob
import kotlinx.coroutines.flow.flowOf
import kotlinx.coroutines.test.TestScope
import kotlinx.coroutines.test.UnconfinedTestDispatcher
import okhttp3.OkHttpClient
import okhttp3.Request
import okhttp3.mockwebserver.Dispatcher
import okhttp3.mockwebserver.MockResponse
import okhttp3.mockwebserver.MockWebServer
import okhttp3.mockwebserver.RecordedRequest
import org.junit.jupiter.api.AfterEach
import org.junit.jupiter.api.BeforeEach
import org.junit.jupiter.api.Test
import kotlin.test.assertEquals
import kotlin.test.assertFalse
import kotlin.test.assertNull
import kotlin.test.assertTrue
@OptIn(ExperimentalCoroutinesApi::class)
class AuthCookieInterceptorTest {
@@ -92,6 +101,87 @@ class AuthCookieInterceptorTest {
assertNull(authStore.sessionCookie.value)
}
@Test
fun `a refused session asks the UI to open sign-in`() {
authStore.setSessionCookie("session=expired")
server.enqueue(MockResponse().setResponseCode(401))
client.newCall(Request.Builder().url(placeholderUrl("/api/test")).build()).execute()
assertTrue(authStore.sessionRejected.value)
}
// A refusal of a request that carried no cookie says nothing about the
// stored login, and must not raise the sign-in screen either.
@Test
fun `a 401 to a request sent without a cookie signs nobody out`() {
server.enqueue(MockResponse().setResponseCode(401))
client.newCall(Request.Builder().url(placeholderUrl("/api/test")).build()).execute()
assertFalse(authStore.sessionRejected.value)
}
// The interceptor reads the cookie when the request starts. A sign-in
// that lands while an old request is in flight must survive that old
// request's 401.
@Test
fun `a 401 for an older cookie keeps the one stored since`() {
authStore.setSessionCookie("session=old")
server.dispatcher =
object : Dispatcher() {
override fun dispatch(request: RecordedRequest): MockResponse {
authStore.setSessionCookie("session=new")
return MockResponse().setResponseCode(401)
}
}
client.newCall(Request.Builder().url(placeholderUrl("/api/test")).build()).execute()
assertEquals("session=new", authStore.sessionCookie.value)
assertFalse(authStore.sessionRejected.value)
}
// At launch the stored cookie loads from the Keystore off the main
// thread. A request made before that finishes waits for it rather than
// going out without the login.
@Test
fun `a request made while the stored cookie is still loading carries it`() {
val slowVault =
object : SessionVault {
override fun read(): String? {
Thread.sleep(SLOW_VAULT_MS)
return "session=stored"
}
override fun write(value: String?): Boolean = true
}
// The store runs on a real background dispatcher here, so its row
// collector lands after construction; the row itself carries the
// server URL so that collector cannot reset it to the default.
val row = AuthSessionEntity(baseUrl = server.url("/").toString())
val dao =
mockk<AuthSessionDao> {
every { observe() } returns flowOf(row)
coEvery { get() } returns row
coEvery { upsert(any()) } returns Unit
coEvery { setSessionCookie(any()) } returns Unit
coEvery { setBaseUrl(any()) } returns Unit
}
val loading = AuthStore(dao, slowVault, CoroutineScope(SupervisorJob() + Dispatchers.Default))
val launchClient =
OkHttpClient.Builder()
.addInterceptor(AuthCookieInterceptor(loading))
.addInterceptor(BaseUrlInterceptor(loading))
.build()
server.enqueue(MockResponse().setResponseCode(200))
launchClient.newCall(Request.Builder().url(placeholderUrl("/api/library/sync")).build()).execute()
assertEquals("session=stored", server.takeRequest().getHeader("Cookie"))
assertEquals("session=stored", loading.sessionCookie.value)
}
@Test
fun `captures Set-Cookie on successful response`() {
server.enqueue(
@@ -145,4 +235,8 @@ class AuthCookieInterceptorTest {
assertEquals("session=keep", authStore.sessionCookie.value)
}
private companion object {
const val SLOW_VAULT_MS = 300L
}
}
@@ -0,0 +1,87 @@
package com.fabledsword.minstrel.auth
import com.fabledsword.minstrel.cache.db.dao.AuthSessionDao
import com.fabledsword.minstrel.cache.db.entities.AuthSessionEntity
import com.jakewharton.retrofit2.converter.kotlinx.serialization.asConverterFactory
import io.mockk.coEvery
import io.mockk.every
import io.mockk.mockk
import kotlinx.coroutines.CoroutineScope
import kotlinx.coroutines.Dispatchers
import kotlinx.coroutines.SupervisorJob
import kotlinx.coroutines.cancel
import kotlinx.coroutines.flow.flowOf
import kotlinx.serialization.json.Json
import okhttp3.MediaType.Companion.toMediaType
import okhttp3.OkHttpClient
import okhttp3.mockwebserver.MockResponse
import okhttp3.mockwebserver.MockWebServer
import org.junit.jupiter.api.AfterEach
import org.junit.jupiter.api.BeforeEach
import org.junit.jupiter.api.Test
import retrofit2.Retrofit
import java.util.concurrent.TimeUnit
import kotlin.test.assertEquals
import kotlin.test.assertNotNull
class AuthControllerTest {
private lateinit var server: MockWebServer
private val scope = CoroutineScope(SupervisorJob() + Dispatchers.Default)
private val json = Json { ignoreUnknownKeys = true }
@BeforeEach
fun setup() {
server = MockWebServer().apply { start() }
}
@AfterEach
fun teardown() {
scope.cancel()
server.shutdown()
}
// At launch the controller refreshes the profile (it carries account
// flags such as debug mode). The stored login loads from the Keystore
// off the main thread, so the refresh has to wait for it rather than
// read the user as signed out and skip.
@Test
fun `the launch profile refresh waits for the stored login`() {
val slowVault =
object : SessionVault {
override fun read(): String? {
Thread.sleep(SLOW_VAULT_MS)
return "session=stored"
}
override fun write(value: String?): Boolean = true
}
val row = AuthSessionEntity(baseUrl = server.url("/").toString())
val dao =
mockk<AuthSessionDao> {
every { observe() } returns flowOf(row)
coEvery { get() } returns row
coEvery { upsert(any()) } returns Unit
coEvery { setSessionCookie(any()) } returns Unit
coEvery { setBaseUrl(any()) } returns Unit
}
val store = AuthStore(dao, slowVault, scope)
val retrofit =
Retrofit.Builder()
.baseUrl(server.url("/"))
.client(OkHttpClient.Builder().build())
.addConverterFactory(json.asConverterFactory("application/json".toMediaType()))
.build()
server.enqueue(MockResponse().setResponseCode(200).setBody("""{"id":"u1","username":"listener"}"""))
AuthController(store, json, scope, retrofit)
val request = server.takeRequest(REQUEST_WAIT_SECONDS, TimeUnit.SECONDS)
assertNotNull(request, "the profile refresh never reached the server")
assertEquals("/api/me", request.path)
}
private companion object {
const val SLOW_VAULT_MS = 300L
const val REQUEST_WAIT_SECONDS = 5L
}
}
@@ -4,6 +4,7 @@ import com.fabledsword.minstrel.models.PlaylistRef
import com.fabledsword.minstrel.models.SystemPlaylistsStatus
import org.junit.jupiter.api.Test
import kotlin.test.assertEquals
import kotlin.test.assertFalse
import kotlin.test.assertTrue
class BuildPlaylistsRowTest {
@@ -103,4 +104,40 @@ class BuildPlaylistsRowTest {
assertTrue(buildSongsLikeRow(emptyList(), SystemPlaylistsStatus(), offline = true).isEmpty())
}
private fun genreMix(name: String, cached: Boolean) = PlaylistRef(
id = name,
userId = "u",
name = name,
systemVariant = "genre_mix",
trackCount = 50,
fullyCached = cached,
)
@Test
fun `genre mixes get their own row in name order and stay out of the carousel`() {
val owned = listOf(
genreMix("Shoegaze mix", cached = true),
genreMix("dub mix", cached = true),
genreMix("Jazz mix", cached = true),
user("u1", cached = true),
)
val row = buildGenreMixRow(owned, offline = false)
val names = row.filterIsInstance<PlaylistRowItem.Real>().map { it.playlist.name }
assertEquals(listOf("dub mix", "Jazz mix", "Shoegaze mix"), names)
val carousel = buildPlaylistsRow(owned, SystemPlaylistsStatus(), offline = false)
.filterIsInstance<PlaylistRowItem.Real>().map { it.playlist.id }
assertEquals(listOf("u1"), carousel)
}
@Test
fun `genre mixes are not refreshable, so a cached one plays offline`() {
assertFalse(genreMix("Jazz mix", cached = true).refreshable)
val owned = listOf(genreMix("A mix", cached = false), genreMix("B mix", cached = true))
val names = buildGenreMixRow(owned, offline = true)
.filterIsInstance<PlaylistRowItem.Real>().map { it.playlist.name }
assertEquals(listOf("B mix", "A mix"), names)
assertTrue(buildGenreMixRow(emptyList(), offline = false).isEmpty())
}
}
@@ -0,0 +1,28 @@
package com.fabledsword.minstrel.update.data
import org.junit.jupiter.api.Test
import kotlin.test.assertEquals
/** Back from the "install unknown apps" page, an update that was asked for carries on. */
class AfterAskingTest {
@Test
fun `allowed now, the update carries on`() {
assertEquals(AfterAsking.FETCH, afterAsking(asked = true, mayInstall = true, offered = true, busy = false))
}
@Test
fun `back without allowing it, the listener is told and the offer stays`() {
assertEquals(AfterAsking.REFUSED, afterAsking(asked = true, mayInstall = false, offered = true, busy = false))
}
@Test
fun `nothing asked, nothing happens`() {
assertEquals(AfterAsking.WAIT, afterAsking(asked = false, mayInstall = true, offered = true, busy = false))
}
@Test
fun `an offer not known yet, or an install under way, waits`() {
assertEquals(AfterAsking.WAIT, afterAsking(asked = true, mayInstall = true, offered = false, busy = false))
assertEquals(AfterAsking.WAIT, afterAsking(asked = true, mayInstall = true, offered = true, busy = true))
}
}
@@ -0,0 +1,120 @@
package com.fabledsword.minstrel.update.data
import com.fabledsword.minstrel.models.wire.UpdateInfoWire
import com.fabledsword.minstrel.update.api.ClientVersionApi
import kotlinx.coroutines.test.runTest
import okhttp3.ResponseBody.Companion.toResponseBody
import org.junit.jupiter.api.Test
import org.junit.jupiter.api.io.TempDir
import retrofit2.HttpException
import retrofit2.Response
import java.io.File
import java.io.IOException
import java.security.MessageDigest
import kotlin.test.assertEquals
import kotlin.test.assertIs
import kotlin.test.assertNotNull
import kotlin.test.assertNull
import kotlin.test.assertTrue
/**
* The fetch worker and the listener's Update both download through here; a
* package only counts as ready once it is the whole build that was offered.
*/
class UpdateRepositoryTest {
@TempDir
lateinit var dir: File
private val apkBytes = "the newer app".toByteArray()
private val sha = MessageDigest.getInstance("SHA-256").digest(apkBytes).joinToString("") { "%02x".format(it) }
private fun wire(code: Long? = 200L, version: String = "2026.10.26.1200", sha256: String = sha) = UpdateInfoWire(
version = version,
code = code,
sizeBytes = apkBytes.size.toLong(),
sha256 = sha256,
)
private var downloads = 0
private fun repo(
version: () -> UpdateInfoWire = { wire() },
body: () -> ByteArray = { apkBytes },
offers: Boolean = true,
) = UpdateRepository(
api = object : ClientVersionApi {
override suspend fun getVersion(): UpdateInfoWire = version()
},
apk = ApkSource {
downloads += 1
body().toResponseBody()
},
dir = dir,
installedCode = 100L,
installedName = "2026.10.20.0900",
offers = offers,
)
@Test
fun `a newer build is offered`() = runTest {
assertIs<Offer.Newer>(repo().check())
}
@Test
fun `the same app built again at a later minute is not offered`() = runTest {
assertEquals(Offer.Current, repo(version = { wire(version = "2026.10.20.0900") }).check())
}
@Test
fun `a server with no app to offer is not a problem`() = runTest {
val none = repo(version = { throw HttpException(Response.error<Any>(404, "".toResponseBody())) })
assertEquals(Offer.Current, none.check())
}
@Test
fun `a server out of reach is a problem, not an all-clear`() = runTest {
assertIs<Offer.Problem>(repo(version = { throw IOException("unreachable") }).check())
}
@Test
fun `a debug build is never offered one`() = runTest {
assertEquals(Offer.Current, repo(offers = false).check())
}
@Test
fun `a build already fetched is installed from disk without a second download`() = runTest {
val r = repo()
val info = (r.check() as Offer.Newer).info
assertIs<Download.Ready>(r.download(info) {})
assertNotNull(r.ready(info))
val again = r.download(info) {}
assertIs<Download.Ready>(again)
assertEquals(1, downloads)
}
@Test
fun `a download cut off partway is not taken for the build`() = runTest {
val r = repo(body = { apkBytes.copyOf(5) })
val info = (r.check() as Offer.Newer).info
assertIs<Download.Problem>(r.download(info) {})
assertNull(r.ready(info))
assertTrue(dir.listFiles().orEmpty().isEmpty(), "the part file is removed")
}
@Test
fun `a download that does not match the digest is not taken for the build`() = runTest {
val r = repo(body = { "the other app".toByteArray() }) // same length, other bytes
val info = (r.check() as Offer.Newer).info
assertIs<Download.Problem>(r.download(info) {})
assertNull(r.ready(info))
}
@Test
fun `tidy drops a package kept for a build no longer offered`() = runTest {
val r = repo()
val info = (r.check() as Offer.Newer).info
r.download(info) {}
r.tidy()
assertNull(r.ready(info))
}
}
@@ -0,0 +1,53 @@
package com.fabledsword.minstrel.update.data
import org.junit.jupiter.api.Test
import kotlin.test.assertEquals
import kotlin.test.assertFalse
import kotlin.test.assertTrue
/** Home tells of each build at most once a day, and says whether it is ready. */
class UpdateToldTest {
private val day = SAY_AGAIN_AFTER_MS
private val t0 = 1_000_000_000L
@Test
fun `a build not yet told of is told at once`() {
assertTrue(dueToSay("3523900", toldBuild = "3523847", toldAt = t0, now = t0 + 1))
assertTrue(dueToSay("3523900", toldBuild = "", toldAt = 0L, now = t0))
}
@Test
fun `the same build is not told again within a day`() {
assertFalse(dueToSay("3523900", toldBuild = "3523900", toldAt = t0, now = t0 + day - 1))
}
@Test
fun `the same build is told again the day after`() {
assertTrue(dueToSay("3523900", toldBuild = "3523900", toldAt = t0, now = t0 + day))
}
@Test
fun `a clock set back tells rather than hiding the update`() {
assertTrue(dueToSay("3523900", toldBuild = "3523900", toldAt = t0, now = t0 - 1))
}
@Test
fun `a build is keyed by its code, or by its name from a server with none`() {
assertEquals("3523900", buildKey(3523900L, "2026.09.10.1432"))
assertEquals("2026.09.10.1432", buildKey(null, "2026.09.10.1432"))
}
@Test
fun `the snackbar says whether the update is already here`() {
assertEquals(
"Version 2026.10.26.1200 of the app is on your server.",
toldWords("2026.10.26.1200", ready = false),
)
assertEquals(
"Version 2026.10.26.1200 of the app is downloaded and ready.",
toldWords("2026.10.26.1200", ready = true),
)
assertEquals("Update", toldAction(ready = false))
assertEquals("Install", toldAction(ready = true))
}
}
@@ -62,6 +62,21 @@ class UpdateVersioningTest {
)
}
/**
* Every server image builds the app again at a later minute, so a CI
* re-run, or a change outside the shipped set, raises the code while the
* app is the same one. The name says so, and it is not offered.
*/
@Test
fun `the same app built again at a later minute is not offered`() {
assertFalse(
isUpdateAvailable(
serverCode = 3523900, serverName = "2026.09.10.1432",
installedCode = 3523847, installedName = "2026.09.10.1432",
),
)
}
@Test
fun `falls back to the name when the server reports no ordering key`() {
assertTrue(
+41 -11
View File
@@ -3,11 +3,17 @@
# Derives the three values a build is stamped with, and the tag that names it.
#
# name=YYYY.MM.DD.HHMM label for people, from the timestamp of the newest
# commit that CHANGED SOMETHING SHIPPED (see SHIPPED)
# commit that CHANGED SOMETHING SHIPPED (see SHIPPED,
# or SHIPPED_APK with --apk)
# code=<int> ordering key, minutes since 2020-01-01 at BUILD time
# tag=v<name> what a release of this commit must be called
#
# Usage: ci/version.sh [<commit-ish>] (default HEAD)
# Usage: ci/version.sh [--apk] [<commit-ish>] (default HEAD)
#
# --apk names the Android app from android/ alone. The app compares this
# name with the one it is running, and a different name is what
# offers an update — so a name that moved for a server-only commit
# would offer every phone a rebuild of the app it already has.
#
# This exists as a script rather than inline workflow YAML for one reason:
# release.yml only runs on `main` and on tags, so anything living inside it is
@@ -30,6 +36,12 @@
set -euo pipefail
readonly EPOCH_2020=1577836800 # 2020-01-01T00:00:00Z
apk=false
if [ "${1:-}" = "--apk" ]; then
apk=true
shift
fi
readonly REF="${1:-HEAD}"
# Both clocks are overridable so a test can pin them. Nothing but tests should
@@ -48,14 +60,17 @@ readonly REF="${1:-HEAD}"
# wrongly included → version moves when nothing shipped. Cosmetic noise in
# a string nobody sorts.
#
# THIS REPO SHIPS TWO ARTIFACTS FROM ONE DERIVATION, and that is why the list
# is shorter than it looks like it should be. The server image ships cmd/,
# internal/, shared/, web/, config.example.yaml and client/; the APK ships
# android/. Neither ships the other's sources — but excluding android/ here
# would stop an Android-only commit from moving the APK's OWN version, which
# is the dangerous direction. So this is the union: exclude only what ships in
# NEITHER, and accept that an Android commit also nudges the server's reported
# version. Over-inclusion across the two, which is the harmless direction.
# THIS LIST NAMES THE SERVER IMAGE, and it is shorter than it looks like it
# should be. The image ships cmd/, internal/, shared/, web/,
# config.example.yaml and client/ — and client/ holds the APK, so android/
# stays in: an Android-only commit changes what the image carries. Exclude
# only what ships in neither artifact; over-inclusion is the harmless
# direction here.
#
# The APK itself is named from SHIPPED_APK (below), where over-inclusion is
# NOT harmless: the app offers an update whenever the name differs from its
# own, so a name moved by a server-only commit offers every phone a rebuild
# of the app it already has.
#
# The family's other repos (roundtable / roundtable-android) each keep a
# tighter list because they are separate repos with one artifact apiece. Do
@@ -100,9 +115,24 @@ readonly SHIPPED=(
':!web/vitest.setup.ts'
)
# What the APK is built from: android/ and nothing else (the Gradle build
# reads no path outside it). Still a denylist within it, for the same reason
# as above — a new directory under android/ counts until named here.
readonly SHIPPED_APK=(
android
':!android/app/src/test' # JVM unit tests
':!android/*.md'
)
if [ "${apk}" = true ]; then
shipped=("${SHIPPED_APK[@]}")
else
shipped=("${SHIPPED[@]}")
fi
commit_epoch="${MINSTREL_COMMIT_EPOCH:-}"
if [ -z "${commit_epoch}" ]; then
commit_epoch="$(git log --format=%ct -1 "${REF}" -- "${SHIPPED[@]}")"
commit_epoch="$(git log --format=%ct -1 "${REF}" -- "${shipped[@]}")"
# Loudly, on purpose. A silent fallback here is the landmine this whole
# script exists to avoid: a plausible-looking version that is quietly wrong,
# on a green run. Realistically this means a shallow clone (no commit in
+67 -1
View File
@@ -16,6 +16,8 @@ import (
"github.com/jackc/pgx/v5/pgxpool"
"git.fabledsword.com/bvandeusen/minstrel/internal/acoustid"
"git.fabledsword.com/bvandeusen/minstrel/internal/backup"
"git.fabledsword.com/bvandeusen/minstrel/internal/buildinfo"
"git.fabledsword.com/bvandeusen/minstrel/internal/catalogue"
"git.fabledsword.com/bvandeusen/minstrel/internal/config"
"git.fabledsword.com/bvandeusen/minstrel/internal/coplay"
@@ -35,6 +37,7 @@ import (
"git.fabledsword.com/bvandeusen/minstrel/internal/requests"
"git.fabledsword.com/bvandeusen/minstrel/internal/scrobble"
"git.fabledsword.com/bvandeusen/minstrel/internal/scrobble/listenbrainz"
"git.fabledsword.com/bvandeusen/minstrel/internal/secret"
"git.fabledsword.com/bvandeusen/minstrel/internal/server"
"git.fabledsword.com/bvandeusen/minstrel/internal/similarity"
"git.fabledsword.com/bvandeusen/minstrel/internal/subsonic"
@@ -58,14 +61,40 @@ func main() {
os.Exit(1)
}
return
case "restore":
if err := runRestore(os.Args[2:]); err != nil {
fmt.Fprintf(os.Stderr, "minstrel restore: %v\n", err)
os.Exit(1)
}
return
}
}
if err := run(); err != nil {
err := run()
if errors.Is(err, errRestart) {
err = restartInPlace()
}
if err != nil {
fmt.Fprintf(os.Stderr, "minstrel: %v\n", err)
os.Exit(1)
}
}
// errRestart is run's answer when the server stopped in order to start again:
// after a web restore, so the restored settings are read at boot.
var errRestart = errors.New("restart requested")
// restartInPlace replaces this process with a fresh start of the same binary,
// arguments and environment. The process id stays the same, so a container
// keeps running without needing a restart policy.
func restartInPlace() error {
exe, err := os.Executable()
if err != nil {
return fmt.Errorf("restart: %w", err)
}
// Exec returns only when it failed.
return fmt.Errorf("restart: %w", syscall.Exec(exe, os.Args, os.Environ()))
}
// safetyNetScanInterval backstops the fsnotify watcher with a low-frequency
// full delta walk, catching anything the watcher missed (inotify watch-limit
// exhaustion, dropped events). Fixed — there is no operator configuration.
@@ -128,6 +157,18 @@ func run() error {
}
defer pool.Close()
// Stored credentials are sealed under a key derived from the stream
// secret. Install it before anything reads one, and seal any value still
// stored in the clear.
secret.Install(cfg.StreamSecret)
sealed, err := secret.SealStored(ctx, pool)
if err != nil {
return fmt.Errorf("seal stored credentials: %w", err)
}
if sealed > 0 {
logger.Info("sealed stored credentials", "count", sealed)
}
// Fingerprinting settings: one instance, shared by the scanner,
// the fingerprint backfill, the duplicate sweep and the admin API, so a save
// reaches all of them without a restart. A load failure is logged, not fatal:
@@ -457,7 +498,20 @@ func run() error {
// The sweeper above holds this same instance, so a save from the admin
// card changes what it does on its next tick.
srv.ReacqSettings = reacqSettings
// The server's own backup: the loop runs the nightly schedule, and the
// admin page starts, tests and configures the same instance.
backups := backup.NewService(pool, logger.With("component", "backup"), buildinfo.Version)
backups.Roots = scanner.Roots
go backups.Loop(ctx)
srv.Backups = backups
srv.StreamSecret = cfg.StreamSecret
restart := make(chan struct{}, 1)
srv.Restart = func() {
select {
case restart <- struct{}{}:
default:
}
}
httpServer := &http.Server{
Addr: cfg.Server.Address,
Handler: srv.Router(),
@@ -483,20 +537,32 @@ func run() error {
close(errCh)
}()
restarting := false
select {
case <-ctx.Done():
logger.Info("shutdown signal received")
case <-restart:
logger.Info("restarting to load the restored backup")
restarting = true
case err := <-errCh:
if err != nil {
return fmt.Errorf("server: %w", err)
}
}
// Ends the background work (scans, workers, schedules) as a signal would.
stop()
shutdownCtx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
defer cancel()
if err := httpServer.Shutdown(shutdownCtx); err != nil {
return fmt.Errorf("shutdown: %w", err)
}
// A backup still running is stopped and records that it was cut off.
backups.Close()
if restarting {
logger.Info("minstrel stopped; starting again")
return errRestart
}
logger.Info("minstrel stopped")
return nil
}
+76
View File
@@ -0,0 +1,76 @@
package main
import (
"context"
"errors"
"flag"
"fmt"
"io"
"os"
"git.fabledsword.com/bvandeusen/minstrel/internal/backup"
"git.fabledsword.com/bvandeusen/minstrel/internal/config"
"git.fabledsword.com/bvandeusen/minstrel/internal/db"
"git.fabledsword.com/bvandeusen/minstrel/internal/logging"
)
const restoreUsage = "usage: minstrel restore [-config PATH] FILE"
// runRestore loads a backup file into a new Minstrel's database: one with no
// accounts and no library yet. It brings the schema up to date first, so a
// backup from an older Minstrel loads into this one. The server isn't running
// meanwhile, so the library scan waits for it to start.
func runRestore(args []string) error {
fs := flag.NewFlagSet("restore", flag.ContinueOnError)
configPath := fs.String("config", os.Getenv("MINSTREL_CONFIG"), "path to YAML config file")
if err := fs.Parse(args); err != nil {
return err
}
if fs.NArg() != 1 {
return errors.New(restoreUsage)
}
cfg, err := config.Load(*configPath)
if err != nil {
return fmt.Errorf("load config: %w", err)
}
if cfg.Database.URL == "" {
return errors.New("no database URL configured (set it in the config file or MINSTREL_DATABASE_URL)")
}
logger, err := logging.New(os.Stderr, cfg.Log.Level, cfg.Log.Format)
if err != nil {
return fmt.Errorf("init logger: %w", err)
}
f, err := os.Open(fs.Arg(0))
if err != nil {
return err
}
defer func() { _ = f.Close() }()
manifest, contents, err := backup.Read(f)
if err != nil {
return err
}
if err := db.Migrate(cfg.Database.URL, logger); err != nil {
return fmt.Errorf("migrate: %w", err)
}
ctx := context.Background()
pool, err := db.Open(ctx, cfg.Database.URL)
if err != nil {
return fmt.Errorf("open db: %w", err)
}
defer pool.Close()
if err := backup.Restore(ctx, pool, contents); err != nil {
return err
}
printRestored(os.Stdout, manifest)
return nil
}
// printRestored says what came back and what happens next.
func printRestored(w io.Writer, m backup.Manifest) {
_, _ = fmt.Fprintf(w, "minstrel: restored the backup made %s by Minstrel %s (%d accounts, %d tracks, %d playlists).\n",
m.CreatedAt.Local().Format("2 Jan 2006 15:04 MST"), m.Build,
m.Rows["users"], m.Rows["tracks"], m.Rows["playlists"])
_, _ = fmt.Fprintln(w, "Start the server. The library scan (at startup, or Scan now under Admin › Library) finds each music file at its path, and the artwork, tags and analysis the backup leaves out are fetched and worked out again.")
_, _ = fmt.Fprintln(w, "Everyone signs in with the password they had. Keys for other services, and the backup's own destination, are entered again in Admin.")
}
+65
View File
@@ -83,6 +83,71 @@ Creating the first account (which becomes the admin) requires that token, so
whoever reaches a newly exposed server first can't claim it. The token
changes on every restart until an account exists.
## Backups
Minstrel can back itself up once a night: accounts, likes, playlists,
listening history, requests and settings, in one file. It is off until an
admin switches it on under **Admin › Backups**. Anything Minstrel can fetch
or work out again (artwork, fingerprints, MusicBrainz data) is left out, and
so are the keys and passwords Minstrel keeps for other services, which are
entered again on a restored server. Sign-in passwords are kept.
A backup goes to one of:
- **an S3 bucket**, path-style (MinIO, Garage, or any S3-compatible store).
The key only needs to add and list objects. Files land under `database/`.
- **a folder** on the server, under `database/` inside it. Mount a volume
there; Minstrel refuses to write into a folder that isn't there.
Minstrel only adds files, named by date, and never deletes one. Keep as many
as you like with the destination's own retention. **Test destination** lists
what is there without writing anything.
**Include the music** also sends the library's files to `music/` in the same
destination, laid out as they are under the library folders. After each
night's database file has landed, Minstrel sends the files it hasn't sent
yet, one at a time, for up to six hours; a large library takes several
nights the first time, and the panel shows how much is in. After that, a
night sends only new files and files that changed. A changed file goes to
its old name, and Minstrel still never deletes anything there, so a file
taken out of the library stays in the backup. On a bucket that keeps
versions or locks objects, set its retention to keep what you want. A file
the bucket refuses (an overwrite a locked object won't take) is listed on
the panel and goes again once it changes or the settings are saved; three
refusals in a row stop that night's music, since that is usually a key that
may not write under `music/`. A bucket's upload limit applies
to the music as well.
### Restoring
A backup restores into a new Minstrel: one nobody has signed up to yet. If
it has already scanned the music, that's fine; the backup's library takes
the place of what the scan found.
1. **Put the music back first**, at the path the library had. The restored
settings name the old library folders, and each track is found again by
its path. If the backup includes the music, copy what is inside `music/`
into the library folder (with several library folders, into the folder
that holds them all).
2. **Load the backup**, either way:
- In the web app, choose **Create account**, then **Restore a backup**.
Pick a `minstrel-….tar` file from the backup's `database/` folder and
enter the setup token from the log (see *First account*). Minstrel
restarts by itself once the backup is in, so it runs on the restored
settings.
- Or, with the server stopped, run
`docker compose run --rm minstrel restore /backups/database/minstrel-20261102T030000Z.tar`,
with the file on a volume the container can read. It brings the database
schema up to date first, so a backup from an older Minstrel loads into a
newer one; a backup from a newer Minstrel is refused.
3. **Sign in** with the password you had. Everyone else's accounts, likes,
playlists and history are back too.
The library scan then finds each file, and the artwork, tags, fingerprints
and loudness are fetched or worked out again over the following hours. Keys
for other services (ListenBrainz, AcoustID, artwork and tag providers,
email, download clients) and the backup's own destination are entered again
under **Admin**. An app that synced with the old server starts its sync over.
## Android app
The app connects over whatever URL you give it, HTTP or HTTPS. Once the
+27 -6
View File
@@ -49,22 +49,43 @@ cross-origin preflight. Requests authenticated with a bearer token or the
Subsonic query parameters carry nothing a browser attaches on its own, so
they aren't checked.
## Subsonic sign-in, and the one password stored in plain text
## Credentials Minstrel has to read back
Some credentials can't be hashed, because Minstrel has to send or compare the
value itself:
- the API keys and passwords for indexers, Prowlarr and download clients;
- the AcoustID key, and the cover-art and tag provider keys;
- the SMTP password;
- the secret key of the S3 bucket backups go to;
- each user's ListenBrainz token and Subsonic password.
These are encrypted in the database (AES-256-GCM). Each column has its own key,
derived from the stream secret (`MINSTREL_STREAM_SECRET`, or the
`stream_secret` file in the data directory), and each value is bound to its
row. The stream secret is never stored in the database, so a copy of the
database alone doesn't reveal any of them. None of them is ever sent back to a
browser: Settings only shows whether one is set.
**Keep the stream secret with the data directory, and don't change it
casually.** If it changes, every stored credential reads as not set, and each
one has to be entered again in Settings (Subsonic passwords generated again).
Minstrel logs a warning for each one that no longer opens.
## Subsonic sign-in
Classic Subsonic clients sign in with `t` and `s`: the MD5 of the password
followed by a random salt. To check that, the server has to know the password
itself, so supporting this sign-in method means storing a password Minstrel
can read. That is the Subsonic password, and it is the only credential
Minstrel keeps unhashed.
can read: the Subsonic password, encrypted as described above.
- **The recommended way in is the API key.** Clients that support the
OpenSubsonic `apiKey` should use it. The key is stored hashed and can be
replaced at any time in Settings.
- **The Subsonic password is never your login password.** Minstrel generates
it (**Settings → Subsonic password**), shows it once, and lets you replace
or turn it off. Because it is random, a copy of the database exposes access
to this server's Subsonic API and nothing else: it can't be a password you
also use somewhere else.
or turn it off. Because it is random, even someone holding both the database
and the stream secret gains access to this server's Subsonic API and nothing
else: it can't be a password you also use somewhere else.
- **`t`/`s` and `p=` sign-in are off for an account until it has a Subsonic
password.** Plain `p=` sign-in is additionally off server-wide unless
`subsonic.allow_plaintext_password` is enabled.
+1 -1
View File
@@ -24,7 +24,7 @@ func Build(ctx context.Context, pool *pgxpool.Pool) (*Service, dbq.AcquisitionSe
if err != nil {
return nil, settings, fmt.Errorf("acquire: read search sources: %w", err)
}
clients, err := q.ListUsableAcquisitionServices(ctx, JobDownload)
clients, err := usableServices(ctx, q, JobDownload)
if err != nil {
return nil, settings, fmt.Errorf("acquire: read download clients: %w", err)
}
+32 -2
View File
@@ -23,6 +23,7 @@ import (
"git.fabledsword.com/bvandeusen/minstrel/internal/exchange"
"git.fabledsword.com/bvandeusen/minstrel/internal/indexer"
"git.fabledsword.com/bvandeusen/minstrel/internal/prowlarr"
"git.fabledsword.com/bvandeusen/minstrel/internal/secret"
)
// Kinds: which list a row appears in.
@@ -159,7 +160,7 @@ func DownloadClient(row dbq.AcquisitionService) (downloader.Driver, error) {
// DownloadClientFor builds the download client a grab went to, by the code
// of its acquisition_services row.
func DownloadClientFor(ctx context.Context, pool *pgxpool.Pool, code string) (downloader.Driver, error) {
row, err := dbq.New(pool).GetAcquisitionService(ctx, code)
row, err := getService(ctx, dbq.New(pool), code)
if err != nil {
if errors.Is(err, pgx.ErrNoRows) {
return nil, fmt.Errorf("the download client %q is no longer set up", code)
@@ -171,7 +172,7 @@ func DownloadClientFor(ctx context.Context, pool *pgxpool.Pool, code string) (do
// Sources builds every enabled search source.
func Sources(ctx context.Context, pool *pgxpool.Pool) ([]Source, error) {
rows, err := dbq.New(pool).ListUsableAcquisitionServices(ctx, JobSearch)
rows, err := usableServices(ctx, dbq.New(pool), JobSearch)
if err != nil {
return nil, err
}
@@ -182,4 +183,33 @@ func Sources(ctx context.Context, pool *pgxpool.Pool) ([]Source, error) {
return out, nil
}
// getService and usableServices are the only reads of a full
// acquisition_services row. They hand back the credentials opened, so the
// clients built from a row are given the key and password themselves.
func getService(ctx context.Context, q *dbq.Queries, code string) (dbq.AcquisitionService, error) {
row, err := q.GetAcquisitionService(ctx, code)
if err != nil {
return row, err
}
return revealed(row), nil
}
func usableServices(ctx context.Context, q *dbq.Queries, job string) ([]dbq.AcquisitionService, error) {
rows, err := q.ListUsableAcquisitionServices(ctx, job)
if err != nil {
return nil, err
}
for i := range rows {
rows[i] = revealed(rows[i])
}
return rows, nil
}
// revealed is row with its sealed credentials opened.
func revealed(row dbq.AcquisitionService) dbq.AcquisitionService {
row.ApiKey = secret.Reveal(secret.AcquisitionAPIKey.Purpose(), row.Code, row.ApiKey)
row.Password = secret.Reveal(secret.AcquisitionPassword.Purpose(), row.Code, row.Password)
return row
}
func msDuration(ms int32) time.Duration { return time.Duration(ms) * time.Millisecond }
+15 -6
View File
@@ -11,6 +11,7 @@ import (
"github.com/jackc/pgx/v5/pgxpool"
"git.fabledsword.com/bvandeusen/minstrel/internal/db/dbq"
"git.fabledsword.com/bvandeusen/minstrel/internal/secret"
)
// ErrNotFound is a code with no row.
@@ -49,7 +50,7 @@ func (s *Store) List(ctx context.Context) ([]dbq.ListAcquisitionServicesRow, err
// Get returns one row, secrets included: for building a client only.
func (s *Store) Get(ctx context.Context, code string) (dbq.AcquisitionService, error) {
row, err := dbq.New(s.pool).GetAcquisitionService(ctx, code)
row, err := getService(ctx, dbq.New(s.pool), code)
if errors.Is(err, pgx.ErrNoRows) {
return row, ErrNotFound
}
@@ -163,19 +164,27 @@ func (s *Store) Delete(ctx context.Context, code string) error {
return nil
}
// SetAPIKey stores a key, trimmed; empty clears it.
// SetAPIKey stores a key, trimmed and sealed; empty clears it.
func (s *Store) SetAPIKey(ctx context.Context, code, key string) error {
sealed, err := secret.Seal(secret.AcquisitionAPIKey.Purpose(), code, strings.TrimSpace(key))
if err != nil {
return err
}
n, err := dbq.New(s.pool).SetAcquisitionServiceAPIKey(ctx, dbq.SetAcquisitionServiceAPIKeyParams{
Code: code, ApiKey: strings.TrimSpace(key),
Code: code, ApiKey: sealed,
})
return affected(n, err)
}
// SetPassword stores a password as typed (spaces are part of a password);
// empty clears it.
// SetPassword stores a password as typed (spaces are part of a password),
// sealed; empty clears it.
func (s *Store) SetPassword(ctx context.Context, code, password string) error {
sealed, err := secret.Seal(secret.AcquisitionPassword.Purpose(), code, password)
if err != nil {
return err
}
n, err := dbq.New(s.pool).SetAcquisitionServicePassword(ctx, dbq.SetAcquisitionServicePasswordParams{
Code: code, Password: password,
Code: code, Password: sealed,
})
return affected(n, err)
}
+10
View File
@@ -12,6 +12,7 @@ import (
"git.fabledsword.com/bvandeusen/minstrel/internal/db"
"git.fabledsword.com/bvandeusen/minstrel/internal/dbtest"
"git.fabledsword.com/bvandeusen/minstrel/internal/secret"
)
func newPool(t *testing.T) *pgxpool.Pool {
@@ -83,6 +84,15 @@ func TestStore_AddsConfiguresAndRemoves_Integration(t *testing.T) {
if err != nil || full.ApiKey != "key" {
t.Errorf("stored key = %q (err %v), want it trimmed", full.ApiKey, err)
}
// The column holds the key sealed, never the key.
var raw string
if err := s.pool.QueryRow(ctx,
"SELECT api_key FROM acquisition_services WHERE code = $1", first.Code).Scan(&raw); err != nil {
t.Fatalf("read api_key: %v", err)
}
if raw == "key" || !secret.IsSealed(raw) {
t.Errorf("api_key column = %q, want it sealed", raw)
}
rows, err := s.List(ctx)
if err != nil || len(rows) != 2 {
t.Fatalf("list: %d rows, err %v", len(rows), err)
+94
View File
@@ -0,0 +1,94 @@
package api
import (
"encoding/json"
"errors"
"net/http"
"strings"
"git.fabledsword.com/bvandeusen/minstrel/internal/apierror"
"git.fabledsword.com/bvandeusen/minstrel/internal/backup"
)
// GET /api/admin/backups — the backup's verdict, settings and recent runs.
func (h *handlers) handleGetBackups(w http.ResponseWriter, r *http.Request) {
st, err := h.backups.Status(r.Context())
if err != nil {
writeErrWithLog(w, h.logger, "admin backups: status", apierror.Internal(err))
return
}
writeJSON(w, http.StatusOK, st)
}
func decodeBackupUpdate(w http.ResponseWriter, r *http.Request) (backup.Update, bool) {
var u backup.Update
if err := json.NewDecoder(r.Body).Decode(&u); err != nil {
writeErr(w, apierror.BadRequest("invalid_body", ""))
return u, false
}
return u, true
}
// fieldErr writes a setting that can't be kept as a 400 whose message says
// which setting and why, and reports whether err was one.
func fieldErr(w http.ResponseWriter, err error) bool {
var fe *backup.FieldError
if !errors.As(err, &fe) {
return false
}
writeErr(w, apierror.BadRequest("invalid_backup_setting", fe.Message))
return true
}
// PUT /api/admin/backups — save the settings. An empty secret_key keeps the
// stored one. Returns the new status.
func (h *handlers) handleUpdateBackups(w http.ResponseWriter, r *http.Request) {
u, ok := decodeBackupUpdate(w, r)
if !ok {
return
}
if err := h.backups.Save(r.Context(), u); err != nil {
if !fieldErr(w, err) {
writeErrWithLog(w, h.logger, "admin backups: save", apierror.Internal(err))
}
return
}
h.handleGetBackups(w, r)
}
// POST /api/admin/backups/test — check the destination in the body without
// saving it, by listing the backups already there. Nothing is written.
func (h *handlers) handleTestBackups(w http.ResponseWriter, r *http.Request) {
u, ok := decodeBackupUpdate(w, r)
if !ok {
return
}
res, err := h.backups.Test(r.Context(), u)
if err != nil {
if fieldErr(w, err) {
return
}
msg := strings.TrimPrefix(err.Error(), "backup: ")
writeErr(w, &apierror.Error{Status: http.StatusBadGateway, Code: "destination_failed", Message: msg})
return
}
writeJSON(w, http.StatusOK, res)
}
// POST /api/admin/backups/run — start a backup now. 202 once it has started;
// 409 while one runs or before the destination is set up.
func (h *handlers) handleRunBackup(w http.ResponseWriter, r *http.Request) {
err := h.backups.Start(r.Context(), backup.TriggerManual)
switch {
case err == nil:
w.WriteHeader(http.StatusAccepted)
case errors.Is(err, backup.ErrRunning):
writeErr(w, apierror.Conflict("backup_running", "A backup is already running."))
case errors.Is(err, backup.ErrNotReady):
writeErr(w, apierror.Conflict("backup_not_ready", "Set up where backups go first."))
case errors.Is(err, backup.ErrClosed):
writeErr(w, apierror.Conflict("shutting_down", "The server is shutting down."))
default:
writeErrWithLog(w, h.logger, "admin backups: start", apierror.Internal(err))
}
}
+10 -5
View File
@@ -10,6 +10,7 @@ import (
"git.fabledsword.com/bvandeusen/minstrel/internal/apierror"
"git.fabledsword.com/bvandeusen/minstrel/internal/db/dbq"
"git.fabledsword.com/bvandeusen/minstrel/internal/mailer"
"git.fabledsword.com/bvandeusen/minstrel/internal/secret"
)
type smtpConfigResp struct {
@@ -76,11 +77,15 @@ func (h *handlers) handleUpdateSMTPConfig(w http.ResponseWriter, r *http.Request
writeErrWithLog(w, h.logger, "admin smtp: get failed", apierror.Internal(err))
return
}
// Preserve existing password when the request sends "" (so the
// admin doesn't have to re-enter it on every save).
password := req.Password
if password == "" {
password = current.Password
// Preserve the stored (sealed) password when the request sends "", so
// the admin doesn't have to re-enter it on every save.
password := current.Password
if req.Password != "" {
password, err = secret.Seal(secret.SMTPPassword.Purpose(), "", req.Password)
if err != nil {
writeErrWithLog(w, h.logger, "admin smtp: seal password", apierror.Internal(err))
return
}
}
if err := q.UpdateSMTPConfig(r.Context(), dbq.UpdateSMTPConfigParams{
Enabled: req.Enabled,
+15 -1
View File
@@ -15,6 +15,7 @@ import (
"git.fabledsword.com/bvandeusen/minstrel/internal/acquire"
"git.fabledsword.com/bvandeusen/minstrel/internal/auth"
"git.fabledsword.com/bvandeusen/minstrel/internal/backup"
"git.fabledsword.com/bvandeusen/minstrel/internal/catalogue"
"git.fabledsword.com/bvandeusen/minstrel/internal/config"
"git.fabledsword.com/bvandeusen/minstrel/internal/coverart"
@@ -40,7 +41,7 @@ import (
// Mount attaches /api/* handlers to r. Public endpoints (login) are outside
// RequireUser; everything else is gated by the middleware. The events writer
// is shared with the Subsonic mount so /rest/scrobble feeds the same store.
func Mount(r chi.Router, pool *pgxpool.Pool, logger *slog.Logger, events *playevents.Writer, recCfg config.RecommendationConfig, recSettings *recsettings.Service, requestSvc *requests.Service, quarantineSvc *quarantine.Service, tracksSvc *tracks.Service, playlistsSvc *playlists.Service, coverEnricher *coverart.Enricher, coverSettings *coverart.SettingsService, tagSettings *tags.SettingsService, scanner *library.Scanner, scanCfg library.RunScanConfig, dataDir string, sender mailer.Sender, bus *eventbus.Bus, playlistScheduler *playlists.Scheduler, streamSecret []byte, netSettings *netsettings.Service, reacqSettings *reacquisition.SettingsService, fpSettings *library.FingerprintSettingsService, loudSettings *library.LoudnessSettingsService, acoustIDLookup *library.AcoustIDLookupWorker) {
func Mount(r chi.Router, pool *pgxpool.Pool, logger *slog.Logger, events *playevents.Writer, recCfg config.RecommendationConfig, recSettings *recsettings.Service, requestSvc *requests.Service, quarantineSvc *quarantine.Service, tracksSvc *tracks.Service, playlistsSvc *playlists.Service, coverEnricher *coverart.Enricher, coverSettings *coverart.SettingsService, tagSettings *tags.SettingsService, scanner *library.Scanner, scanCfg library.RunScanConfig, dataDir string, sender mailer.Sender, bus *eventbus.Bus, playlistScheduler *playlists.Scheduler, streamSecret []byte, netSettings *netsettings.Service, reacqSettings *reacquisition.SettingsService, fpSettings *library.FingerprintSettingsService, loudSettings *library.LoudnessSettingsService, acoustIDLookup *library.AcoustIDLookupWorker, backups *backup.Service, restart func()) {
rng := rand.New(rand.NewSource(rand.Int63()))
setupToken, err := auth.NewSetupToken()
if err != nil {
@@ -74,6 +75,8 @@ func Mount(r chi.Router, pool *pgxpool.Pool, logger *slog.Logger, events *playev
fingerprintSettings: fpSettings,
loudnessSettings: loudSettings,
acoustIDLookup: acoustIDLookup,
backups: backups,
restart: restart,
acquisition: acquire.NewStore(pool),
catalogue: cat,
wanted: wanted.New(pool, cat, logger),
@@ -97,6 +100,7 @@ func Mount(r chi.Router, pool *pgxpool.Pool, logger *slog.Logger, events *playev
api.Post("/auth/login", h.handleLogin)
api.Post("/auth/register", h.handleRegister)
api.Get("/auth/setup-status", h.handleSetupStatus)
api.Post("/auth/restore", h.handleRestore)
api.Post("/auth/forgot-password", h.handleForgotPassword)
api.Post("/auth/reset-password", h.handleResetPassword)
@@ -358,6 +362,11 @@ func Mount(r chi.Router, pool *pgxpool.Pool, logger *slog.Logger, events *playev
admin.Post("/tag-sources/{provider_id}/test", h.handleTestTagSource)
admin.Post("/tag-sources/research", h.handleResearchTags)
admin.Get("/backups", h.handleGetBackups)
admin.Put("/backups", h.handleUpdateBackups)
admin.Post("/backups/test", h.handleTestBackups)
admin.Post("/backups/run", h.handleRunBackup)
admin.Get("/smtp-config", h.handleGetSMTPConfig)
admin.Put("/smtp-config", h.handleUpdateSMTPConfig)
admin.Post("/smtp-config/test", h.handleTestSMTPConfig)
@@ -429,6 +438,11 @@ type handlers struct {
// acoustIDLookup is the AcoustID lookup worker: its settings, its
// status and "look up now". Nil serves the defaults (off).
acoustIDLookup *library.AcoustIDLookupWorker
// backups is the server's own backup: settings, status, test and run.
backups *backup.Service
// restart stops the server and starts it again in place, so a restored
// backup's settings are read at boot. Nil in tests.
restart func()
// acquisition is the configured indexers and download clients.
acquisition *acquire.Store
// catalogue is the release catalogue read from MusicBrainz, and
+94
View File
@@ -0,0 +1,94 @@
package api
import (
"errors"
"net/http"
"git.fabledsword.com/bvandeusen/minstrel/internal/apierror"
"git.fabledsword.com/bvandeusen/minstrel/internal/backup"
"git.fabledsword.com/bvandeusen/minstrel/internal/db/dbq"
)
// SetupTokenHeader carries the setup token on a restore upload, whose body is
// the backup file itself.
const SetupTokenHeader = "X-Setup-Token"
type restoreResp struct {
// Accounts is how many accounts came back; everyone signs in with the
// password they had.
Accounts int64 `json:"accounts"`
// Restarting says the server restarts now to load the restored settings.
// The scan at its start finds each music file and fills in what the
// backup leaves out.
Restarting bool `json:"restarting"`
}
// handleRestore implements POST /api/auth/restore: a new server, with no
// accounts yet, loads a backup file sent as the request body. It asks for the
// setup token from the server log, as creating the first account does, since
// the backup's admins become this server's admins. A library the server's
// first scan found is replaced by the backup's.
//
// Then the server restarts. Settings are read once, at boot, so the restored
// ones (the library folders among them) take effect the same way they do
// after `minstrel restore`, and the startup scan finds each file at its path.
//
// 200 restored · 400 not_a_backup · 403 setup_token_invalid ·
// 409 not_empty (accounts already) · 409 newer_backup
func (h *handlers) handleRestore(w http.ResponseWriter, r *http.Request) {
ctx := r.Context()
q := dbq.New(h.pool)
n, err := q.CountUsers(ctx)
if err != nil {
writeErrWithLog(w, h.logger, "restore: count users", apierror.Internal(err))
return
}
if n > 0 {
writeErr(w, apierror.Conflict("not_empty", "this server already has accounts; a backup restores into a new one"))
return
}
if h.requireSetupToken && !h.setupToken.Matches(r.Header.Get(SetupTokenHeader)) {
h.logger.Warn("restore: needs the setup token", "setup_token", h.setupToken.Value())
writeErr(w, apierror.Forbidden("setup_token_invalid", "the setup token from the server log is required to restore a backup"))
return
}
manifest, contents, err := backup.Read(r.Body)
if err != nil {
var tooBig *http.MaxBytesError
if errors.As(err, &tooBig) {
writeErr(w, &apierror.Error{Status: http.StatusRequestEntityTooLarge, Code: "too_large", Message: "the file is larger than a backup can be"})
return
}
writeErr(w, apierror.BadRequest("not_a_backup", "that file isn't a Minstrel backup"))
return
}
switch err := backup.Restore(ctx, h.pool, contents); {
case errors.Is(err, backup.ErrNotEmpty):
writeErr(w, apierror.Conflict("not_empty", "this server already has accounts; a backup restores into a new one"))
return
case errors.Is(err, backup.ErrNewerSchema):
writeErr(w, apierror.Conflict("newer_backup", "the backup was made by a newer Minstrel; update this server first"))
return
case errors.Is(err, backup.ErrNotABackup):
writeErr(w, apierror.BadRequest("not_a_backup", "that file isn't a Minstrel backup"))
return
case err != nil:
writeErrWithLog(w, h.logger, "restore: load the backup", apierror.Internal(err))
return
}
out := restoreResp{}
if out.Accounts, err = q.CountUsers(ctx); err != nil {
h.logger.Warn("restore: count restored accounts", "err", err)
}
h.logger.Info("restored a backup", "made_at", manifest.CreatedAt, "build", manifest.Build,
"schema", manifest.Schema, "accounts", out.Accounts)
out.Restarting = h.restart != nil
writeJSON(w, http.StatusOK, out)
if h.restart != nil {
// The server drains this request before it stops, so the answer
// reaches the browser first.
h.restart()
}
}
+98
View File
@@ -0,0 +1,98 @@
package api
import (
"bytes"
"context"
"encoding/json"
"net/http"
"net/http/httptest"
"strings"
"testing"
"time"
"git.fabledsword.com/bvandeusen/minstrel/internal/auth"
"git.fabledsword.com/bvandeusen/minstrel/internal/backup"
"git.fabledsword.com/bvandeusen/minstrel/internal/dbtest"
)
// A new server takes a backup only with the setup token from its log, only
// while it has no accounts, and only a file that is a backup; then the
// backup's accounts sign in again.
func TestRestore_ANewServerTakesABackupWithTheSetupToken(t *testing.T) {
h, pool := testHandlers(t)
ctx := context.Background()
truncateLibrary(t, pool)
resetUsers(t, h)
seedUser(t, pool, "restored", "abcd1234", true)
c, err := backup.Snapshot(ctx, pool)
if err != nil {
t.Fatal(err)
}
var file bytes.Buffer
if _, err := backup.Write(&file, c, time.Now(), "test"); err != nil {
t.Fatal(err)
}
resetUsers(t, h)
token, err := auth.NewSetupToken()
if err != nil {
t.Fatal(err)
}
h.setupToken = token
h.requireSetupToken = true
restarts := 0
h.restart = func() { restarts++ }
restore := func(setupToken string, body []byte) *httptest.ResponseRecorder {
req := httptest.NewRequest(http.MethodPost, "/api/auth/restore", bytes.NewReader(body))
req.Header.Set("Content-Type", "application/x-tar")
if setupToken != "" {
req.Header.Set(SetupTokenHeader, setupToken)
}
rec := httptest.NewRecorder()
h.handleRestore(rec, req)
return rec
}
if w := restore("", file.Bytes()); w.Code != http.StatusForbidden {
t.Errorf("no setup token: status = %d, want 403", w.Code)
}
if w := restore("wrong", file.Bytes()); w.Code != http.StatusForbidden {
t.Errorf("a wrong setup token: status = %d, want 403", w.Code)
}
if w := restore(token.Value(), []byte("not a tar file")); w.Code != http.StatusBadRequest ||
!strings.Contains(w.Body.String(), "not_a_backup") {
t.Errorf("not a backup: status = %d body = %s, want 400 not_a_backup", w.Code, w.Body.String())
}
var n int
if err := pool.QueryRow(ctx, "SELECT count(*) FROM users").Scan(&n); err != nil || n != 0 {
t.Fatalf("a refused restore left %d accounts (%v)", n, err)
}
if restarts != 0 {
t.Errorf("a refused restore restarted the server %d times", restarts)
}
w := restore(token.Value(), file.Bytes())
if w.Code != http.StatusOK {
t.Fatalf("restore: status = %d body = %s", w.Code, w.Body.String())
}
var out restoreResp
if err := json.NewDecoder(w.Body).Decode(&out); err != nil || out.Accounts != 1 || !out.Restarting {
t.Errorf("restore answered %+v (%v), want one account and a restart", out, err)
}
if restarts != 1 {
t.Errorf("the restore restarted the server %d times, want once so the restored settings load", restarts)
}
var admin bool
if err := pool.QueryRow(ctx, "SELECT is_admin FROM users WHERE username = $1",
dbtest.TestUserPrefix+"restored").Scan(&admin); err != nil || !admin {
t.Errorf("the restored admin: is_admin = %v, %v", admin, err)
}
if w := restore(token.Value(), file.Bytes()); w.Code != http.StatusConflict ||
!strings.Contains(w.Body.String(), "not_empty") {
t.Errorf("a second restore: status = %d body = %s, want 409 not_empty", w.Code, w.Body.String())
}
if restarts != 1 {
t.Errorf("a refused second restore restarted the server (%d restarts)", restarts)
}
}
+50
View File
@@ -17,8 +17,11 @@ package api
// client treats 404 as "no update channel available."
import (
"crypto/sha256"
"encoding/hex"
"encoding/json"
"errors"
"io"
"net/http"
"os"
"path/filepath"
@@ -62,6 +65,41 @@ var (
clientAPKLastDownloadMu sync.Mutex
)
// clientAPKSum is the bundled APK's sha256, kept until the file changes
// under it (another size or modification time), so a version check every
// few minutes from every phone does not re-read tens of megabytes each time.
type clientAPKSum struct {
size int64
mod time.Time
hex string
}
var (
clientAPKSumCache clientAPKSum
clientAPKSumCacheMu sync.Mutex
)
// clientAPKDigest returns the sha256 of the APK at path, in hex.
func clientAPKDigest(path string, st os.FileInfo) (string, error) {
clientAPKSumCacheMu.Lock()
defer clientAPKSumCacheMu.Unlock()
c := clientAPKSumCache
if c.hex != "" && c.size == st.Size() && c.mod.Equal(st.ModTime()) {
return c.hex, nil
}
f, err := os.Open(path)
if err != nil {
return "", err
}
defer func() { _ = f.Close() }()
h := sha256.New()
if _, err := io.Copy(h, f); err != nil {
return "", err
}
clientAPKSumCache = clientAPKSum{size: st.Size(), mod: st.ModTime(), hex: hex.EncodeToString(h.Sum(nil))}
return clientAPKSumCache.hex, nil
}
func testResetClientAPKRateLimit() {
clientAPKLastDownloadMu.Lock()
defer clientAPKLastDownloadMu.Unlock()
@@ -115,6 +153,11 @@ type clientVersionResponse struct {
Channel string `json:"channel,omitempty"`
APKURL string `json:"apk_url"`
SizeBytes int64 `json:"size_bytes"`
// SHA256 is worked out from the file being served, not copied from the
// sidecar, so it cannot disagree with the bytes the phone receives. The
// app downloads ahead on Wi-Fi and installs later; this is how it knows
// the file it kept is the whole build that was offered.
SHA256 string `json:"sha256"`
}
// handleClientVersion returns the bundled Android client version + a
@@ -159,12 +202,19 @@ func (h *handlers) handleClientVersion(w http.ResponseWriter, _ *http.Request) {
return
}
sum, err := clientAPKDigest(apkPath, stat)
if err != nil {
writeErrWithLog(w, h.logger, "client_version: digest apk", err)
return
}
writeJSON(w, http.StatusOK, clientVersionResponse{
Version: strings.TrimSpace(sidecar.Name),
Code: sidecar.Code,
Channel: strings.TrimSpace(sidecar.Channel),
APKURL: "/api/client/apk",
SizeBytes: stat.Size(),
SHA256: sum,
})
}
+35
View File
@@ -2,6 +2,8 @@ package api
import (
"context"
"crypto/sha256"
"encoding/hex"
"encoding/json"
"io"
"log/slog"
@@ -128,6 +130,39 @@ func TestClientVersion_200WithBothFiles(t *testing.T) {
if resp.SizeBytes != size {
t.Errorf("size_bytes: want %d, got %d", size, resp.SizeBytes)
}
want := sha256.Sum256([]byte("fake apk content"))
if resp.SHA256 != hex.EncodeToString(want[:]) {
t.Errorf("sha256: want the served file's digest %x, got %q", want, resp.SHA256)
}
}
// The digest is cached by size and modification time; a new APK dropped in
// place must be answered with its own digest, not the old one's.
func TestClientVersion_DigestFollowsAReplacedAPK(t *testing.T) {
writeClientAssets(t, `{"name":"a","code":1}`)
first := decodeClientVersion(t, getClientVersion(t))
dir := os.Getenv("MINSTREL_CLIENT_APK_DIR")
apk := filepath.Join(dir, clientAPKFilename)
if err := os.WriteFile(apk, []byte("another, longer build"), 0o644); err != nil {
t.Fatal(err)
}
second := decodeClientVersion(t, getClientVersion(t))
want := sha256.Sum256([]byte("another, longer build"))
if second.SHA256 != hex.EncodeToString(want[:]) || second.SHA256 == first.SHA256 {
t.Errorf("sha256 after replace: want %x, got %q (before: %q)", want, second.SHA256, first.SHA256)
}
}
func decodeClientVersion(t *testing.T, rr *httptest.ResponseRecorder) clientVersionResponse {
t.Helper()
if rr.Code != http.StatusOK {
t.Fatalf("want 200, got %d (body: %s)", rr.Code, rr.Body.String())
}
var resp clientVersionResponse
if err := json.Unmarshal(rr.Body.Bytes(), &resp); err != nil {
t.Fatal(err)
}
return resp
}
// A release published before ordering keys were recorded has a name and
+27 -7
View File
@@ -36,7 +36,9 @@ type syncResponse struct {
// Responses:
// - 204 No Content — no changes since cursor
// - 200 OK — JSON syncResponse with upserts, deletes, new cursor
// - 410 Gone — cursor older than oldest log row; client must reset
// - 410 Gone — the client must reset its cursor to 0: the cursor is
// older than a delete the log has dropped, or past anything this server
// has handed out
// - 401 / 500 — standard envelope errors
func (h *handlers) handleLibrarySync(w http.ResponseWriter, r *http.Request) {
user, ok := requireUser(w, r)
@@ -52,16 +54,17 @@ func (h *handlers) handleLibrarySync(w http.ResponseWriter, r *http.Request) {
q := dbq.New(h.pool)
// 410 path: if the requested cursor is older than the oldest row,
// tell the client to reset. (No compaction job in this slice; the
// path exists for future compaction support.)
// The log keeps every entity's latest change, so a client syncing from
// 0 gets the whole library. It drops deletes older than its retention,
// and a cursor below the newest dropped one may have missed it: the
// client resets and syncs again from 0.
if since > 0 {
minCursor, err := q.GetMinLibraryChangeCursor(ctx)
trimmed, err := q.GetLibraryChangesTrimmedThrough(ctx)
if err != nil {
writeErrWithLog(w, h.logger, "library_sync: GetMinLibraryChangeCursor", err)
writeErrWithLog(w, h.logger, "library_sync: GetLibraryChangesTrimmedThrough", err)
return
}
if minCursor > 0 && since+1 < minCursor {
if since < trimmed {
writeErr(w, &apierror.Error{
Status: http.StatusGone,
Code: "cursor_too_old",
@@ -69,6 +72,23 @@ func (h *handlers) handleLibrarySync(w http.ResponseWriter, r *http.Request) {
})
return
}
// A cursor past anything this server has handed out comes from a
// server restored from an older backup: the changes between the
// backup and the cursor are gone, and the ones made from here on would
// reuse those ids. The app resets the same way.
lastCursor, err := q.GetLastIssuedLibraryChangeCursor(ctx)
if err != nil {
writeErrWithLog(w, h.logger, "library_sync: GetLastIssuedLibraryChangeCursor", err)
return
}
if since > lastCursor {
writeErr(w, &apierror.Error{
Status: http.StatusGone,
Code: "cursor_ahead",
Message: "this server holds no changes up to that cursor; reset client cursor to 0 and resync",
})
return
}
}
changes, err := q.GetLibraryChangesSince(ctx, dbq.GetLibraryChangesSinceParams{
+81
View File
@@ -6,6 +6,7 @@ import (
"net/http"
"net/http/httptest"
"strconv"
"strings"
"testing"
"git.fabledsword.com/bvandeusen/minstrel/internal/auth"
@@ -204,3 +205,83 @@ func TestSplitOnce(t *testing.T) {
}
}
}
// A server restored from an older backup has handed out fewer cursors than an
// app remembers; the app is told to reset rather than wait for ids it would
// then skip. A compacted log is not that: the cursor the server handed out
// still counts.
func TestLibrarySync_ACursorPastTheServersLastChangeIsReset(t *testing.T) {
h, pool := testHandlers(t)
truncateLibrary(t, pool)
u := seedUser(t, pool, "alice", "x", false)
ctx := context.Background()
var artistID string
if err := pool.QueryRow(ctx,
`INSERT INTO artists (name, sort_name) VALUES ('A', 'A') RETURNING id::text`).Scan(&artistID); err != nil {
t.Fatal(err)
}
tx, err := pool.Begin(ctx)
if err != nil {
t.Fatal(err)
}
if err := sync.LogChange(ctx, tx, sync.EntityArtist, artistID, sync.OpUpsert); err != nil {
t.Fatal(err)
}
if err := tx.Commit(ctx); err != nil {
t.Fatal(err)
}
var last int64
if err := pool.QueryRow(ctx, `SELECT max(id) FROM library_changes`).Scan(&last); err != nil {
t.Fatal(err)
}
w := callLibrarySync(h, u, strconv.FormatInt(last+50, 10))
if w.Code != http.StatusGone || !strings.Contains(w.Body.String(), "cursor_ahead") {
t.Errorf("a cursor past the last change: status = %d body = %s, want 410 cursor_ahead", w.Code, w.Body.String())
}
// The compactor trims every row; the app's cursor is still one this
// server handed out.
if _, err := pool.Exec(ctx, `DELETE FROM library_changes`); err != nil {
t.Fatal(err)
}
if w := callLibrarySync(h, u, strconv.FormatInt(last, 10)); w.Code != http.StatusNoContent {
t.Errorf("the last cursor after compaction: status = %d body = %s, want 204", w.Code, w.Body.String())
}
}
// An app is sent back to 0 only when its cursor is below a delete the log has
// dropped, which it may have missed. A cursor at that delete has seen it, and
// one between dropped and kept rows replays what is still there.
func TestLibrarySync_ACursorBelowADroppedDeleteIsReset(t *testing.T) {
h, pool := testHandlers(t)
truncateLibrary(t, pool)
u := seedUser(t, pool, "alice", "x", false)
ctx := context.Background()
for i := 0; i < 5; i++ {
if _, err := pool.Exec(ctx, `INSERT INTO library_changes (entity_type, entity_id, op) VALUES ('artist', $1, 'upsert')`,
strconv.Itoa(i)); err != nil {
t.Fatal(err)
}
}
var dropped int64
if err := pool.QueryRow(ctx, `SELECT min(id) + 2 FROM library_changes`).Scan(&dropped); err != nil {
t.Fatal(err)
}
if _, err := pool.Exec(ctx, `UPDATE library_changes_trimmed SET through = $1`, dropped); err != nil {
t.Fatal(err)
}
if w := callLibrarySync(h, u, strconv.FormatInt(dropped-1, 10)); w.Code != http.StatusGone ||
!strings.Contains(w.Body.String(), "cursor_too_old") {
t.Errorf("a cursor below the dropped delete: status = %d body = %s, want 410 cursor_too_old", w.Code, w.Body.String())
}
if w := callLibrarySync(h, u, strconv.FormatInt(dropped, 10)); w.Code != http.StatusOK {
t.Errorf("a cursor at the dropped delete: status = %d body = %s, want 200", w.Code, w.Body.String())
}
if w := callLibrarySync(h, u, "0"); w.Code != http.StatusOK {
t.Errorf("a sync from 0: status = %d body = %s, want 200", w.Code, w.Body.String())
}
}
+2 -1
View File
@@ -13,6 +13,7 @@ import (
"github.com/go-chi/chi/v5"
"git.fabledsword.com/bvandeusen/minstrel/internal/auth"
"git.fabledsword.com/bvandeusen/minstrel/internal/backup"
"git.fabledsword.com/bvandeusen/minstrel/internal/config"
"git.fabledsword.com/bvandeusen/minstrel/internal/db/dbq"
"git.fabledsword.com/bvandeusen/minstrel/internal/eventbus"
@@ -467,7 +468,7 @@ func TestRoutesRegisteredInMount(t *testing.T) {
r := chi.NewRouter()
w := playevents.NewWriter(h.pool, slog.New(slog.NewTextHandler(io.Discard, nil)),
30*time.Minute, 0.5, 30000)
Mount(r, h.pool, h.logger, w, config.RecommendationConfig{RadioSize: 50, RadioSizeMax: 200, RecentlyPlayedHours: 1}, h.recSettings, h.requests, h.quarantine, h.tracks, h.playlists, h.coverart, h.coverSettings, h.tagSettings, h.scanner, h.scanCfg, h.dataDir, nil, eventbus.New(), nil, nil, h.netSettings, nil, nil, nil, nil)
Mount(r, h.pool, h.logger, w, config.RecommendationConfig{RadioSize: 50, RadioSizeMax: 200, RecentlyPlayedHours: 1}, h.recSettings, h.requests, h.quarantine, h.tracks, h.playlists, h.coverart, h.coverSettings, h.tagSettings, h.scanner, h.scanCfg, h.dataDir, nil, eventbus.New(), nil, nil, h.netSettings, nil, nil, nil, nil, backup.NewService(h.pool, h.logger, "test"), nil)
paths := []string{
"/api/artists",
+6 -4
View File
@@ -11,6 +11,7 @@ import (
"git.fabledsword.com/bvandeusen/minstrel/internal/apierror"
"git.fabledsword.com/bvandeusen/minstrel/internal/db/dbq"
"git.fabledsword.com/bvandeusen/minstrel/internal/secret"
)
// LBStatus is the GET response body and the PUT response body. Token is
@@ -53,10 +54,11 @@ func (h *handlers) handlePutListenBrainz(w http.ResponseWriter, r *http.Request)
q := dbq.New(h.pool)
if body.Token != nil {
t := *body.Token
var tokenPtr *string
if t != "" {
tokenPtr = &t
tokenPtr, err := secret.SealPtr(secret.ListenBrainzToken.Purpose(), user.ID.String(), body.Token)
if err != nil {
h.logger.Error("api: seal listenbrainz token", "err", err)
writeErr(w, apierror.InternalMsg("update failed", err))
return
}
if err := q.SetListenBrainzToken(r.Context(), dbq.SetListenBrainzTokenParams{
ID: user.ID, ListenbrainzToken: tokenPtr,
@@ -88,6 +88,8 @@ func bucketRecSource(src string) recFamily {
return recFamily{"for_you", "For You", intentGoTo}
case src == "songs_like_artist":
return recFamily{"songs_like_artist", "Songs like…", intentGoTo}
case src == "genre_mix":
return recFamily{"genre_mix", "Genre mixes", intentGoTo}
case src == "radio" || strings.HasPrefix(src, "radio:"):
return recFamily{"radio", "Radio", intentGoTo}
case src == "discover":
+8 -1
View File
@@ -8,6 +8,7 @@ import (
"git.fabledsword.com/bvandeusen/minstrel/internal/apierror"
"git.fabledsword.com/bvandeusen/minstrel/internal/audit"
"git.fabledsword.com/bvandeusen/minstrel/internal/db/dbq"
"git.fabledsword.com/bvandeusen/minstrel/internal/secret"
)
// The Subsonic password is for clients that sign in with Subsonic's t/s
@@ -51,9 +52,15 @@ func (h *handlers) handleGenerateMySubsonicPassword(w http.ResponseWriter, r *ht
return
}
pw := base64.RawURLEncoding.EncodeToString(b)
sealed, err := secret.Seal(secret.SubsonicPassword.Purpose(), user.ID.String(), pw)
if err != nil {
h.logger.Error("generate subsonic password: seal failed", "err", err)
writeErr(w, apierror.Internal(err))
return
}
if err := dbq.New(h.pool).SetSubsonicPassword(r.Context(), dbq.SetSubsonicPasswordParams{
ID: user.ID,
SubsonicPassword: &pw,
SubsonicPassword: &sealed,
}); err != nil {
h.logger.Error("generate subsonic password: update failed", "err", err)
writeErr(w, apierror.Internal(err))
+14 -1
View File
@@ -11,6 +11,7 @@ import (
"github.com/go-chi/chi/v5"
"git.fabledsword.com/bvandeusen/minstrel/internal/db/dbq"
"git.fabledsword.com/bvandeusen/minstrel/internal/secret"
)
func newMeSubsonicPasswordRouter(h *handlers) chi.Router {
@@ -21,6 +22,8 @@ func newMeSubsonicPasswordRouter(h *handlers) chi.Router {
return r
}
// readSubsonicPassword returns the stored Subsonic password opened. It fails
// the test if the column holds the password in the clear.
func readSubsonicPassword(t *testing.T, h *handlers, user dbq.User) *string {
t.Helper()
var pw *string
@@ -28,7 +31,17 @@ func readSubsonicPassword(t *testing.T, h *handlers, user dbq.User) *string {
"SELECT subsonic_password FROM users WHERE id = $1", user.ID).Scan(&pw); err != nil {
t.Fatalf("read subsonic_password: %v", err)
}
return pw
if pw == nil {
return nil
}
if !secret.IsSealed(*pw) {
t.Fatalf("subsonic_password is stored in the clear")
}
plain, err := secret.Open(secret.SubsonicPassword.Purpose(), user.ID.String(), *pw)
if err != nil {
t.Fatalf("open subsonic_password: %v", err)
}
return &plain
}
func TestSubsonicPassword_GenerateIsRandomAndNotTheLoginPassword(t *testing.T) {
+226
View File
@@ -0,0 +1,226 @@
package backup
import (
"archive/tar"
"bytes"
"encoding/json"
"errors"
"fmt"
"io"
"os"
"path/filepath"
"slices"
"strings"
"time"
)
// The backup file is laid out so that what didn't change between two days is
// the same bytes both times, which suits a deduplicating off-site copy such as
// Fabled Archives' restic:
//
// - Never compressed. Compression rewrites every byte after the first
// change, so no chunk would match again.
// - One member per table, tables/<table>.jsonl: one row per line, in
// primary-key order, keys sorted, no spacing.
// - What changes every run lives only in manifest.json, the first member.
// Member headers are fixed (mtime 0, owner 0, mode 0644).
// - A new dated name each run, so the target keeps a history and a sender
// only ever adds.
const (
FormatName = "minstrel-backup-tar"
FormatVersion = 1
manifestName = "manifest.json"
tablesDir = "tables/"
tableSuffix = ".jsonl"
namePrefix = "minstrel-"
nameSuffix = ".tar"
// maxMember bounds any one member a read takes in: far past the largest
// library's tracks table, small enough that a hostile file can't fill
// memory.
maxMember = 1 << 30
)
// Manifest is the file's first member.
type Manifest struct {
Format string `json:"format"`
FormatVersion int `json:"format_version"`
CreatedAt time.Time `json:"created_at"`
// Build is the Minstrel version that wrote the file.
Build string `json:"build"`
// Schema is the database's migration version when the file was written.
// A restore refuses a file from a newer schema than its own.
Schema int64 `json:"schema"`
// Rows is each table's row count, so a read can tell a whole member from
// a cut-off one.
Rows map[string]int `json:"rows"`
}
// FileName is a backup's file name: minstrel-20261102T030000Z.tar, in UTC, so
// names sort by time.
func FileName(at time.Time) string {
return namePrefix + at.UTC().Format("20060102T150405Z") + nameSuffix
}
// IsFileName reports whether a name is one FileName gives.
func IsFileName(name string) bool {
stamp, ok := strings.CutPrefix(name, namePrefix)
if !ok {
return false
}
stamp, ok = strings.CutSuffix(stamp, nameSuffix)
if !ok {
return false
}
_, err := time.Parse("20060102T150405Z", stamp)
return err == nil
}
// Write writes c as a backup file to w and returns its manifest.
func Write(w io.Writer, c Contents, at time.Time, build string) (Manifest, error) {
names := make([]string, 0, len(c.Tables))
m := Manifest{
Format: FormatName,
FormatVersion: FormatVersion,
CreatedAt: at.UTC(),
Build: build,
Schema: c.Schema,
Rows: make(map[string]int, len(c.Tables)),
}
for name, lines := range c.Tables {
names = append(names, name)
m.Rows[name] = len(lines)
}
slices.Sort(names)
manifest, err := json.MarshalIndent(m, "", " ")
if err != nil {
return m, err
}
tw := tar.NewWriter(w)
if err := member(tw, manifestName, append(manifest, '\n')); err != nil {
return m, err
}
for _, name := range names {
var body bytes.Buffer
for _, line := range c.Tables[name] {
body.Write(line)
body.WriteByte('\n')
}
if err := member(tw, tablesDir+name+tableSuffix, body.Bytes()); err != nil {
return m, err
}
}
return m, tw.Close()
}
// WriteFile writes c into dir under its dated name. The bytes go to a hidden
// .part file first, are synced, and only then take the backup's name, so
// anything watching the folder never sees half a backup under a backup's
// name, and a run that fails leaves nothing behind.
func WriteFile(dir string, c Contents, at time.Time, build string) (string, Manifest, error) {
final := filepath.Join(dir, FileName(at))
part := filepath.Join(dir, "."+FileName(at)+".part")
f, err := os.OpenFile(part, os.O_WRONLY|os.O_CREATE|os.O_TRUNC, 0o600)
if err != nil {
return "", Manifest{}, fmt.Errorf("backup: create %s: %w", part, err)
}
m, err := Write(f, c, at, build)
if err == nil {
err = f.Sync()
}
if cerr := f.Close(); err == nil {
err = cerr
}
if err == nil {
err = os.Rename(part, final)
}
if err != nil {
_ = os.Remove(part)
return "", m, fmt.Errorf("backup: write %s: %w", final, err)
}
if d, err := os.Open(dir); err == nil {
_ = d.Sync()
_ = d.Close()
}
return final, m, nil
}
func member(tw *tar.Writer, name string, data []byte) error {
hdr := &tar.Header{
Name: name,
Mode: 0o644,
Size: int64(len(data)),
ModTime: time.Unix(0, 0),
Typeflag: tar.TypeReg,
Format: tar.FormatUSTAR,
}
if err := tw.WriteHeader(hdr); err != nil {
return err
}
_, err := tw.Write(data)
return err
}
// Read reads a backup file, refusing anything that isn't a whole one.
func Read(r io.Reader) (Manifest, Contents, error) {
var m Manifest
c := Contents{Tables: map[string][][]byte{}}
tr := tar.NewReader(r)
first := true
for {
hdr, err := tr.Next()
if errors.Is(err, io.EOF) {
break
}
if err != nil {
return m, c, fmt.Errorf("%w: %w", ErrNotABackup, err)
}
if hdr.Size > maxMember {
return m, c, fmt.Errorf("%w: %s is too large", ErrNotABackup, hdr.Name)
}
data, err := io.ReadAll(io.LimitReader(tr, maxMember))
if err != nil {
return m, c, fmt.Errorf("%w: %w", ErrNotABackup, err)
}
if first {
first = false
if hdr.Name != manifestName || json.Unmarshal(data, &m) != nil || m.Format != FormatName {
return m, c, ErrNotABackup
}
if m.FormatVersion > FormatVersion {
return m, c, fmt.Errorf("%w: it was written by a newer Minstrel (format %d)", ErrNotABackup, m.FormatVersion)
}
c.Schema = m.Schema
continue
}
table, ok := strings.CutPrefix(hdr.Name, tablesDir)
if !ok {
continue
}
if table, ok = strings.CutSuffix(table, tableSuffix); !ok || table == "" {
continue
}
var lines [][]byte
for line := range bytes.SplitSeq(data, []byte("\n")) {
if len(line) == 0 {
continue
}
if !json.Valid(line) {
return m, c, fmt.Errorf("%w: a row in %s is not JSON", ErrNotABackup, table)
}
lines = append(lines, line)
}
c.Tables[table] = lines
}
if first {
return m, c, ErrNotABackup
}
for table, want := range m.Rows {
if got := len(c.Tables[table]); got != want {
return m, c, fmt.Errorf("%w: %s holds %d rows, the manifest says %d", ErrNotABackup, table, got, want)
}
}
return m, c, nil
}
+194
View File
@@ -0,0 +1,194 @@
package backup
import (
"archive/tar"
"bytes"
"errors"
"io"
"os"
"path/filepath"
"testing"
"time"
)
func sample() Contents {
return Contents{
Schema: 98,
Tables: map[string][][]byte{
"users": {[]byte(`{"id":"u1","username":"ada"}`), []byte(`{"id":"u2","username":"bo"}`)},
"tracks": {[]byte(`{"file_path":"/music/a.flac","id":"t1"}`)},
"wants": {},
},
}
}
func members(t *testing.T, file []byte) map[string][]byte {
t.Helper()
out := map[string][]byte{}
tr := tar.NewReader(bytes.NewReader(file))
for {
hdr, err := tr.Next()
if errors.Is(err, io.EOF) {
return out
}
if err != nil {
t.Fatal(err)
}
data, _ := io.ReadAll(tr)
out[hdr.Name] = data
}
}
func TestABackupFileReadsBackAsWritten(t *testing.T) {
var buf bytes.Buffer
at := time.Date(2026, 11, 2, 3, 0, 0, 0, time.UTC)
if _, err := Write(&buf, sample(), at, "2026.11.02.0300"); err != nil {
t.Fatal(err)
}
m, c, err := Read(&buf)
if err != nil {
t.Fatal(err)
}
if m.Schema != 98 || c.Schema != 98 || m.Build != "2026.11.02.0300" || !m.CreatedAt.Equal(at) {
t.Errorf("manifest = %+v", m)
}
want := sample()
for name, lines := range want.Tables {
got := c.Tables[name]
if len(got) != len(lines) {
t.Errorf("%s: %d rows, want %d", name, len(got), len(lines))
continue
}
for i := range lines {
if !bytes.Equal(got[i], lines[i]) {
t.Errorf("%s row %d = %s, want %s", name, i, got[i], lines[i])
}
}
}
}
// Two days over unchanged data differ only in the manifest, so a
// deduplicating copy stores the tables once.
func TestOnlyTheManifestChangesBetweenDays(t *testing.T) {
var day1, day2 bytes.Buffer
if _, err := Write(&day1, sample(), time.Date(2026, 11, 1, 3, 0, 0, 0, time.UTC), "a"); err != nil {
t.Fatal(err)
}
if _, err := Write(&day2, sample(), time.Date(2026, 11, 2, 3, 0, 0, 0, time.UTC), "b"); err != nil {
t.Fatal(err)
}
a, b := members(t, day1.Bytes()), members(t, day2.Bytes())
if len(a) != 4 {
t.Fatalf("members = %d, want the manifest and three tables", len(a))
}
for name, data := range a {
same := bytes.Equal(data, b[name])
if name == manifestName && same {
t.Error("the manifest is identical across days; it should carry the run's time")
}
if name != manifestName && !same {
t.Errorf("%s differs between days over the same rows", name)
}
}
// The table members' bytes, headers included, sit unchanged in both files.
tracks := a[tablesDir+"tracks"+tableSuffix]
if !bytes.Contains(day1.Bytes(), tracks) || !bytes.Contains(day2.Bytes(), tracks) {
t.Error("the tracks member is not stored verbatim")
}
}
func TestReadRefusesWhatIsNotAWholeBackup(t *testing.T) {
var good bytes.Buffer
if _, err := Write(&good, sample(), time.Now(), "x"); err != nil {
t.Fatal(err)
}
tarOf := func(name string, data []byte) []byte {
var buf bytes.Buffer
tw := tar.NewWriter(&buf)
_ = member(tw, name, data)
_ = tw.Close()
return buf.Bytes()
}
short := func() []byte {
// The manifest promises two users; the member holds one.
var buf bytes.Buffer
tw := tar.NewWriter(&buf)
_ = member(tw, manifestName, []byte(`{"format":"minstrel-backup-tar","format_version":1,"rows":{"users":2}}`))
_ = member(tw, tablesDir+"users"+tableSuffix, []byte(`{"id":"u1"}`+"\n"))
_ = tw.Close()
return buf.Bytes()
}
cases := map[string][]byte{
"empty": nil,
"not a tar": []byte("hello"),
"cut off": good.Bytes()[:good.Len()/2],
"no manifest": tarOf(tablesDir+"users"+tableSuffix, []byte(`{"id":"u1"}`)),
"another format": tarOf(manifestName, []byte(`{"format":"fablednotary-backup-tar","format_version":1}`)),
"a newer format": tarOf(manifestName, []byte(`{"format":"minstrel-backup-tar","format_version":99}`)),
"a member short": short(),
}
for name, file := range cases {
if _, _, err := Read(bytes.NewReader(file)); !errors.Is(err, ErrNotABackup) {
t.Errorf("%s: err = %v, want ErrNotABackup", name, err)
}
}
}
func TestWriteFileLandsUnderTheDatedNameOnly(t *testing.T) {
dir := t.TempDir()
at := time.Date(2026, 11, 2, 3, 0, 0, 0, time.UTC)
path, _, err := WriteFile(dir, sample(), at, "x")
if err != nil {
t.Fatal(err)
}
if filepath.Base(path) != "minstrel-20261102T030000Z.tar" {
t.Errorf("path = %s", path)
}
entries, _ := os.ReadDir(dir)
if len(entries) != 1 || entries[0].Name() != filepath.Base(path) {
t.Errorf("the folder holds %v, want only the backup", entries)
}
f, err := os.Open(path)
if err != nil {
t.Fatal(err)
}
defer func() { _ = f.Close() }()
if _, _, err := Read(f); err != nil {
t.Errorf("the written file doesn't read: %v", err)
}
}
func TestAFailedWriteFileLeavesNothingBehind(t *testing.T) {
dir := filepath.Join(t.TempDir(), "missing")
if _, _, err := WriteFile(dir, sample(), time.Now(), "x"); err == nil {
t.Fatal("writing into a missing folder succeeded")
}
if _, err := os.Stat(dir); !os.IsNotExist(err) {
t.Errorf("the failed write created %s", dir)
}
}
func TestFileNamesSortByTimeAndAreRecognised(t *testing.T) {
earlier := FileName(time.Date(2026, 1, 9, 23, 0, 0, 0, time.UTC))
// 01:00 at UTC-5 is 06:00 UTC: the name is in UTC, not the clock's zone.
later := FileName(time.Date(2026, 1, 10, 1, 0, 0, 0, time.FixedZone("x", -5*3600)))
if later != "minstrel-20260110T060000Z.tar" {
t.Errorf("name = %s, want it in UTC", later)
}
if earlier >= later {
t.Errorf("%s does not sort before %s", earlier, later)
}
for name, want := range map[string]bool{
earlier: true,
"minstrel-20261102T030000Z.tar": true,
".minstrel-20261102T030000Z.tar.part": false,
"minstrel-yesterday.tar": false,
"fablednotary-20261102T030000Z.tar": false,
} {
if IsFileName(name) != want {
t.Errorf("IsFileName(%q) = %v, want %v", name, !want, want)
}
}
}
+256
View File
@@ -0,0 +1,256 @@
package backup_test
import (
"bytes"
"context"
"encoding/json"
"errors"
"io"
"log/slog"
"os"
"slices"
"testing"
"github.com/jackc/pgx/v5"
"github.com/jackc/pgx/v5/pgxpool"
"git.fabledsword.com/bvandeusen/minstrel/internal/backup"
"git.fabledsword.com/bvandeusen/minstrel/internal/db"
"git.fabledsword.com/bvandeusen/minstrel/internal/dbtest"
)
func newPool(t *testing.T) *pgxpool.Pool {
t.Helper()
if testing.Short() {
t.Skip("skipping integration test in -short mode")
}
dsn := os.Getenv("MINSTREL_TEST_DATABASE_URL")
if dsn == "" {
t.Skip("MINSTREL_TEST_DATABASE_URL not set")
}
if err := db.Migrate(dsn, slog.New(slog.NewTextHandler(io.Discard, nil))); err != nil {
t.Fatalf("migrate: %v", err)
}
pool, err := pgxpool.New(context.Background(), dsn)
if err != nil {
t.Fatalf("pool: %v", err)
}
t.Cleanup(pool.Close)
dbtest.ResetDB(t, pool)
return pool
}
func index(name string) int {
return slices.IndexFunc(backup.Tables, func(t backup.Table) bool { return t.Name == name })
}
// Every table in the schema is backed up or excluded with a reason, so a new
// table fails here until someone decides which.
func TestEveryTableIsAccountedFor_Integration(t *testing.T) {
pool := newPool(t)
rows, err := pool.Query(context.Background(), "SELECT tablename FROM pg_tables WHERE schemaname = 'public'")
if err != nil {
t.Fatal(err)
}
schema, err := pgx.CollectRows(rows, pgx.RowTo[string])
if err != nil {
t.Fatal(err)
}
for _, p := range backup.Coverage(schema) {
t.Error(p)
}
}
// A restore loads each table after the ones it references, and never needs a
// table the backup leaves out.
func TestTablesComeAfterWhatTheyReference_Integration(t *testing.T) {
pool := newPool(t)
rows, err := pool.Query(context.Background(), `
SELECT cl.relname, pc.relname
FROM pg_constraint co
JOIN pg_class cl ON cl.oid = co.conrelid
JOIN pg_class pc ON pc.oid = co.confrelid
JOIN pg_namespace n ON n.oid = cl.relnamespace
WHERE co.contype = 'f' AND n.nspname = 'public'`)
if err != nil {
t.Fatal(err)
}
type fk struct{ child, parent string }
fks, err := pgx.CollectRows(rows, func(r pgx.CollectableRow) (fk, error) {
var f fk
err := r.Scan(&f.child, &f.parent)
return f, err
})
if err != nil {
t.Fatal(err)
}
if len(fks) == 0 {
t.Fatal("found no foreign keys; the query is wrong")
}
for _, f := range fks {
child, parent := index(f.child), index(f.parent)
if child < 0 {
continue
}
switch {
case parent < 0:
t.Errorf("%s is backed up but references %s, which is excluded", f.child, f.parent)
case parent >= child && f.parent != f.child:
t.Errorf("%s references %s, so it must come after it in Tables", f.child, f.parent)
}
}
}
// Every omitted column exists, so a renamed column fails here rather than a
// sealed credential slipping into the file under its new name.
func TestOmittedColumnsExist_Integration(t *testing.T) {
pool := newPool(t)
for _, tbl := range backup.Tables {
for _, col := range tbl.Omit {
var n int
if err := pool.QueryRow(context.Background(), `
SELECT count(*) FROM information_schema.columns
WHERE table_schema = 'public' AND table_name = $1 AND column_name = $2`,
tbl.Name, col).Scan(&n); err != nil || n != 1 {
t.Errorf("%s.%s: found %d columns, err %v", tbl.Name, col, n, err)
}
}
}
}
func mustExec(t *testing.T, pool *pgxpool.Pool, sql string, args ...any) {
t.Helper()
if _, err := pool.Exec(context.Background(), sql, args...); err != nil {
t.Fatalf("%s: %v", sql, err)
}
}
func mustID(t *testing.T, pool *pgxpool.Pool, sql string, args ...any) string {
t.Helper()
var id string
if err := pool.QueryRow(context.Background(), sql, args...).Scan(&id); err != nil {
t.Fatalf("%s: %v", sql, err)
}
return id
}
// A backup restored into an emptied instance gives back the same rows, byte
// for byte, without the credentials and markers it leaves out.
func TestARestoredInstanceHoldsWhatTheBackupHeld_Integration(t *testing.T) {
pool := newPool(t)
ctx := context.Background()
user := mustID(t, pool, `INSERT INTO users (username, password_hash, api_token_hash, listenbrainz_token)
VALUES ($1, 'hash', 'backup-token', 'lb-token') RETURNING id::text`, dbtest.TestUserPrefix+"backup")
artist := mustID(t, pool, `INSERT INTO artists (name, sort_name, artist_art_source) VALUES ('Artist', 'artist', 'none') RETURNING id::text`)
album := mustID(t, pool, `INSERT INTO albums (title, sort_title, artist_id, cover_art_source, cover_art_path)
VALUES ('Album', 'album', $1, 'mbcaa', '/data/covers/a.jpg') RETURNING id::text`, artist)
track := mustID(t, pool, `INSERT INTO tracks (title, album_id, artist_id, duration_ms, file_path, file_size, file_format, tag_source)
VALUES ('Track', $1, $2, 180000, '/music/a.flac', 1000, 'flac', 'musicbrainz') RETURNING id::text`, album, artist)
session := mustID(t, pool, `INSERT INTO play_sessions (user_id, started_at, last_event_at) VALUES ($1, now(), now()) RETURNING id::text`, user)
mustExec(t, pool, `INSERT INTO play_events (user_id, track_id, session_id, started_at) VALUES ($1, $2, $3, now())`, user, track, session)
mustExec(t, pool, `INSERT INTO general_likes (user_id, track_id) VALUES ($1, $2)`, user, track)
playlist := mustID(t, pool, `INSERT INTO playlists (user_id, name) VALUES ($1, 'Mine') RETURNING id::text`, user)
mustExec(t, pool, `INSERT INTO playlist_tracks (playlist_id, position, track_id, title, artist_name, album_title, duration_sec)
VALUES ($1, 0, $2, 'Track', 'Artist', 'Album', 180)`, playlist, track)
mustExec(t, pool, `INSERT INTO follows (artist_mbid, artist_name, profile_id, followed_by)
SELECT 'mbid-1', 'Artist', id, $1 FROM release_type_profiles WHERE is_default`, user)
mustExec(t, pool, `INSERT INTO library_changes (entity_type, entity_id, op) VALUES ('track', $1, 'upsert')`, track)
first, err := backup.Snapshot(ctx, pool)
if err != nil {
t.Fatalf("snapshot: %v", err)
}
for _, tbl := range backup.Tables {
for _, line := range first.Tables[tbl.Name] {
var row map[string]json.RawMessage
if err := json.Unmarshal(line, &row); err != nil {
t.Fatalf("%s: %v", tbl.Name, err)
}
for _, col := range tbl.Omit {
if _, ok := row[col]; ok {
t.Errorf("%s.%s is in the backup", tbl.Name, col)
}
}
}
}
for _, tbl := range []string{"users", "tracks", "play_events", "general_likes", "playlist_tracks", "follows", "library_changes"} {
if len(first.Tables[tbl]) == 0 {
t.Errorf("the backup holds no %s", tbl)
}
}
if err := backup.Restore(ctx, pool, first); !errors.Is(err, backup.ErrNotEmpty) {
t.Fatalf("restore over data: err = %v, want ErrNotEmpty", err)
}
newer := first
newer.Schema++
if err := backup.Restore(ctx, pool, newer); !errors.Is(err, backup.ErrNewerSchema) {
t.Fatalf("restore from a newer schema: err = %v, want ErrNewerSchema", err)
}
// A new server scans its music before anyone signs up, so it already has
// a library, here with a track the backup doesn't know. It still takes
// the backup: the backup's library replaces what the scan found, which the
// next scan finds again under the backup's ids.
mustExec(t, pool, "TRUNCATE users CASCADE")
scannedArtist := mustID(t, pool, `INSERT INTO artists (name, sort_name) VALUES ('Scanned', 'scanned') RETURNING id::text`)
scannedAlbum := mustID(t, pool, `INSERT INTO albums (title, sort_title, artist_id) VALUES ('Scanned', 'scanned', $1) RETURNING id::text`, scannedArtist)
scanned := mustID(t, pool, `INSERT INTO tracks (title, album_id, artist_id, duration_ms, file_path, file_size, file_format)
VALUES ('Scanned', $1, $2, 1000, '/music/b.flac', 10, 'flac') RETURNING id::text`, scannedAlbum, scannedArtist)
mustExec(t, pool, `INSERT INTO library_changes (entity_type, entity_id, op) VALUES ('track', $1, 'upsert')`, scanned)
if err := backup.Restore(ctx, pool, first); err != nil {
t.Fatalf("restore: %v", err)
}
second, err := backup.Snapshot(ctx, pool)
if err != nil {
t.Fatalf("snapshot after restore: %v", err)
}
for _, tbl := range backup.Tables {
a, b := first.Tables[tbl.Name], second.Tables[tbl.Name]
if len(a) != len(b) {
t.Errorf("%s: %d rows restored, the backup held %d", tbl.Name, len(b), len(a))
continue
}
for i := range a {
if !bytes.Equal(a[i], b[i]) {
t.Errorf("%s row %d:\nbacked up %s\nrestored %s", tbl.Name, i+1, a[i], b[i])
}
}
}
// What the backup leaves out comes back unset, so the server asks for the
// credential again and redoes the art and tags.
var lbToken, coverSource, tagSource *string
if err := pool.QueryRow(ctx, `SELECT u.listenbrainz_token, al.cover_art_source, tr.tag_source
FROM users u, albums al, tracks tr WHERE u.id = $1 AND al.id = $2 AND tr.id = $3`,
user, album, track).Scan(&lbToken, &coverSource, &tagSource); err != nil {
t.Fatal(err)
}
if lbToken != nil || coverSource != nil || tagSource != nil {
t.Errorf("token %v, cover source %v, tag source %v; want all unset", lbToken, coverSource, tagSource)
}
// The change feed carries on after the restored ids.
var restoredMax, issued, next int64
if err := pool.QueryRow(ctx, "SELECT max(id) FROM library_changes").Scan(&restoredMax); err != nil {
t.Fatal(err)
}
// The sequence says the restored history ends there, which is what the
// library sync compares an app's cursor with.
if err := pool.QueryRow(ctx, `SELECT COALESCE(pg_sequence_last_value(
pg_get_serial_sequence('library_changes', 'id')::regclass), 0)`).Scan(&issued); err != nil {
t.Fatal(err)
}
if issued != restoredMax {
t.Errorf("the change sequence stands at %d, want the restored %d", issued, restoredMax)
}
if err := pool.QueryRow(ctx, `INSERT INTO library_changes (entity_type, entity_id, op)
VALUES ('track', $1, 'upsert') RETURNING id`, track).Scan(&next); err != nil {
t.Fatal(err)
}
if next <= restoredMax {
t.Errorf("next change id %d, want it after the restored %d", next, restoredMax)
}
}
+281
View File
@@ -0,0 +1,281 @@
package backup
import (
"context"
"crypto/sha256"
"encoding/hex"
"errors"
"fmt"
"io"
"os"
"path/filepath"
"strings"
"time"
"github.com/jackc/pgx/v5/pgtype"
"git.fabledsword.com/bvandeusen/minstrel/internal/db/dbq"
)
const (
// The music's folder inside the destination, beside database/.
musicDir = "music"
// MusicBudget is how long one run may spend sending music. A library
// bigger than a night's worth carries on from where it stopped on the
// next run, so the first copy of a large library spans several nights.
MusicBudget = 6 * time.Hour
musicBatch = 200
// refusedInARow stops a pass: one refusal is that file, several in a row
// are the key or the bucket refusing the music as a whole.
refusedInARow = 3
)
// MusicProgress is how much of the library the backup holds as it is now.
type MusicProgress struct {
TotalFiles int64 `json:"total_files"`
TotalBytes int64 `json:"total_bytes"`
SentFiles int64 `json:"sent_files"`
SentBytes int64 `json:"sent_bytes"`
RefusedFiles int64 `json:"refused_files"`
Refused []RefusedFile `json:"refused"`
}
// RefusedFile is a library file the destination would not take, with its
// answer. It is tried again once the file changes.
type RefusedFile struct {
Path string `json:"path"`
Error string `json:"error"`
At time.Time `json:"at"`
}
// musicSent is what one run's music pass did.
type musicSent struct {
files int
bytes int64
note string
}
// commonRoot is the deepest folder holding every library root, so the music's
// paths in the backup are the same whichever root a file is under.
func commonRoot(roots []string) string {
if len(roots) == 0 {
return ""
}
common := strings.Split(filepath.Clean(roots[0]), string(filepath.Separator))
for _, r := range roots[1:] {
parts := strings.Split(filepath.Clean(r), string(filepath.Separator))
n := 0
for n < len(common) && n < len(parts) && common[n] == parts[n] {
n++
}
common = common[:n]
}
root := strings.Join(common, string(filepath.Separator))
if root == "" {
return string(filepath.Separator)
}
return root
}
// musicPath is file's path inside music/, relative to root, with forward
// slashes. A file outside root keeps its whole path.
func musicPath(root, file string) string {
rel, err := filepath.Rel(root, file)
if root == "" || err != nil || rel == ".." || strings.HasPrefix(rel, ".."+string(filepath.Separator)) {
rel = strings.TrimPrefix(filepath.Clean(file), string(filepath.Separator))
}
return musicDir + "/" + filepath.ToSlash(rel)
}
func (s *Service) musicProgress(ctx context.Context) (*MusicProgress, error) {
q := dbq.New(s.pool)
p, err := q.BackupMusicProgress(ctx)
if err != nil {
return nil, fmt.Errorf("backup: read music progress: %w", err)
}
refused, err := q.RefusedBackupMusic(ctx, recentRuns)
if err != nil {
return nil, fmt.Errorf("backup: list refused music: %w", err)
}
out := &MusicProgress{
TotalFiles: p.TotalFiles, TotalBytes: p.TotalBytes,
SentFiles: p.SentFiles, SentBytes: p.SentBytes,
RefusedFiles: p.RefusedFiles, Refused: make([]RefusedFile, 0, len(refused)),
}
for _, r := range refused {
out.Refused = append(out.Refused, RefusedFile{Path: r.FilePath, Error: r.Error, At: r.CheckedAt.Time})
}
return out, nil
}
// sendMusic sends library files the destination doesn't hold as they are now,
// one at a time in path order, until all are sent or ctx ends. Every file is
// recorded as it lands, so a run cut short loses nothing. It never deletes
// anything at the destination.
func (s *Service) sendMusic(ctx context.Context, st dbq.BackupSetting) musicSent {
var out musicSent
q := dbq.New(s.pool)
var roots []string
if s.Roots != nil {
roots = s.Roots()
}
root := commonRoot(roots)
var bucket Bucket
if st.Destination == DestinationS3 {
b, err := bucketOf(st)
if err != nil {
out.note = describe(err)
return out
}
bucket = b
}
after := ""
refused := 0
for {
rows, err := q.PendingBackupMusic(ctx, dbq.PendingBackupMusicParams{AfterPath: after, MaxRows: musicBatch})
if err != nil {
out.note = musicStopped(ctx, err)
return out
}
if len(rows) == 0 {
return out
}
for _, r := range rows {
after = r.FilePath
if err := ctx.Err(); err != nil {
out.note = musicStopped(ctx, err)
return out
}
info, err := os.Stat(r.FilePath)
if err != nil || !info.Mode().IsRegular() {
// Gone or unreadable: the library scan settles that, and the
// file is looked at again on the next run.
continue
}
mtime := info.ModTime().Truncate(time.Microsecond)
key := musicPath(root, r.FilePath)
location := key
if st.Destination == DestinationFolder {
location = filepath.Join(st.Folder, filepath.FromSlash(key))
}
rec := dbq.RecordBackupMusicParams{
TrackID: r.ID, Size: info.Size(),
Mtime: pgtype.Timestamptz{Time: mtime, Valid: true},
Location: location,
}
if r.SentSize != nil && *r.SentSize == info.Size() && r.SentMtime.Valid && r.SentMtime.Time.Equal(mtime) &&
(r.SentError == nil || *r.SentError == "") {
// Touched by a scan but the same file: nothing to send.
if err := q.RecordBackupMusic(ctx, rec); err != nil {
out.note = musicStopped(ctx, err)
return out
}
continue
}
if st.Destination == DestinationFolder {
err = copyFile(r.FilePath, location, mtime)
} else {
err = putFile(ctx, bucket, key, r.FilePath, int64(st.UploadLimitKib)*1024)
}
switch {
case err == nil:
out.files++
out.bytes += info.Size()
refused = 0
case ObjectRefused(err):
rec.Error = strings.TrimPrefix(err.Error(), "backup: ")
if refused++; refused >= refusedInARow {
out.note = fmt.Sprintf("Music stopped: the destination refused %d files in a row (%s). "+
"Check the key may write under %s/; refused files go again once you save the settings.",
refused, rec.Error, musicDir)
}
case ctx.Err() == nil && errors.Is(err, errSourceRead):
// The library file couldn't be read through: looked at again next run.
continue
default:
out.note = musicStopped(ctx, err)
return out
}
if err := q.RecordBackupMusic(ctx, rec); err != nil {
out.note = musicStopped(ctx, err)
return out
}
if out.note != "" {
return out
}
}
}
}
// musicStopped is why a music pass ended early, as the panel shows it.
func musicStopped(ctx context.Context, err error) string {
switch {
case errors.Is(ctx.Err(), context.DeadlineExceeded):
return fmt.Sprintf("Music stopped for tonight after %d hours; it carries on with the next backup.", int(MusicBudget/time.Hour))
case ctx.Err() != nil:
return "Music stopped because the server stopped; it carries on with the next backup."
}
return "Music stopped: " + describe(err) + " It carries on with the next backup."
}
// errSourceRead marks a library file that could not be read.
var errSourceRead = errors.New("backup: the library file could not be read")
// putFile uploads file to key. It reads the file twice, once to hash it
// (S3 signs over the body's hash) and once to send it through the limit.
func putFile(ctx context.Context, b Bucket, key, file string, perSecond int64) error {
f, err := os.Open(file)
if err != nil {
return fmt.Errorf("%w: %v", errSourceRead, err)
}
defer func() { _ = f.Close() }()
h := sha256.New()
size, err := io.Copy(h, f)
if err != nil {
return fmt.Errorf("%w: %v", errSourceRead, err)
}
if _, err := f.Seek(0, io.SeekStart); err != nil {
return fmt.Errorf("%w: %v", errSourceRead, err)
}
return b.Put(ctx, key, Throttle(ctx, f, perSecond), size, hex.EncodeToString(h.Sum(nil)))
}
// copyFile copies file to dst under a hidden .part name first, so the folder
// never holds half a file under the file's own name. The copy keeps the
// file's modification time. The upload limit is the bucket's: a folder copy
// runs at the disk's pace.
func copyFile(file, dst string, mtime time.Time) error {
in, err := os.Open(file)
if err != nil {
return fmt.Errorf("%w: %v", errSourceRead, err)
}
defer func() { _ = in.Close() }()
if err := os.MkdirAll(filepath.Dir(dst), 0o700); err != nil {
return fmt.Errorf("backup: create %s: %w", filepath.Dir(dst), err)
}
part := filepath.Join(filepath.Dir(dst), "."+filepath.Base(dst)+".part")
out, err := os.OpenFile(part, os.O_WRONLY|os.O_CREATE|os.O_TRUNC, 0o600)
if err != nil {
return fmt.Errorf("backup: create %s: %w", part, err)
}
_, err = io.Copy(out, in)
if err == nil {
err = out.Sync()
}
if cerr := out.Close(); err == nil {
err = cerr
}
if err == nil {
err = os.Chtimes(part, mtime, mtime)
}
if err == nil {
err = os.Rename(part, dst)
}
if err != nil {
_ = os.Remove(part)
return fmt.Errorf("backup: write %s: %w", dst, err)
}
return nil
}
+217
View File
@@ -0,0 +1,217 @@
package backup_test
import (
"context"
"net/http"
"net/http/httptest"
"os"
"path/filepath"
"strings"
"sync/atomic"
"testing"
"time"
"github.com/jackc/pgx/v5/pgxpool"
"git.fabledsword.com/bvandeusen/minstrel/internal/backup"
)
// library writes music files under a new library folder and adds a track for
// each. It returns the folder.
func library(t *testing.T, pool *pgxpool.Pool, files map[string]string) string {
t.Helper()
root := t.TempDir()
artist := mustID(t, pool, `INSERT INTO artists (name, sort_name) VALUES ('Artist', 'artist') RETURNING id::text`)
album := mustID(t, pool, `INSERT INTO albums (title, sort_title, artist_id) VALUES ('Album', 'album', $1) RETURNING id::text`, artist)
for rel, body := range files {
path := filepath.Join(root, filepath.FromSlash(rel))
if err := os.MkdirAll(filepath.Dir(path), 0o755); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(path, []byte(body), 0o644); err != nil {
t.Fatal(err)
}
mustExec(t, pool, `INSERT INTO tracks (title, album_id, artist_id, duration_ms, file_path, file_size, file_format)
VALUES ($1, $2, $3, 1000, $4, $5, 'flac')`, rel, album, artist, path, len(body))
}
return root
}
func runOnce(t *testing.T, svc *backup.Service) backup.Run {
t.Helper()
if err := svc.Start(context.Background(), backup.TriggerManual); err != nil {
t.Fatal(err)
}
waitIdle(t, svc)
st := status(t, svc)
if len(st.Runs) == 0 || st.Runs[0].Status != "succeeded" {
t.Fatalf("the run didn't succeed: %+v", st.Runs)
}
return st.Runs[0]
}
func TestTheMusicGoesToTheFolderOnceAndAgainOnlyWhenItChanges_Integration(t *testing.T) {
svc, pool := newService(t)
ctx := context.Background()
lib := library(t, pool, map[string]string{"A/one.flac": "one", "B/two.flac": "second"})
svc.Roots = func() []string { return []string{lib} }
dest := t.TempDir()
if err := svc.Save(ctx, backup.Update{Enabled: true, Destination: backup.DestinationFolder, Folder: dest, RunHour: 3, IncludeMusic: true}); err != nil {
t.Fatal(err)
}
first := runOnce(t, svc)
if first.MusicFiles != 2 || first.MusicBytes != int64(len("one")+len("second")) || first.MusicNote != "" {
t.Errorf("the first run sent %d files, %d bytes, note %q", first.MusicFiles, first.MusicBytes, first.MusicNote)
}
got, err := os.ReadFile(filepath.Join(dest, "music", "B", "two.flac"))
if err != nil || string(got) != "second" {
t.Errorf("music/B/two.flac = %q, %v", got, err)
}
src, _ := os.Stat(filepath.Join(lib, "A", "one.flac"))
dst, err := os.Stat(filepath.Join(dest, "music", "A", "one.flac"))
// The database keeps microseconds, so the copy's time is the file's to the µs.
if err != nil || !dst.ModTime().Equal(src.ModTime().Truncate(time.Microsecond)) {
t.Errorf("the copy's time = %v, want the file's %v (%v)", dst, src.ModTime(), err)
}
if p := status(t, svc).Music; p == nil || p.TotalFiles != 2 || p.SentFiles != 2 || p.RefusedFiles != 0 {
t.Errorf("progress = %+v, want 2 of 2", p)
}
if again := runOnce(t, svc); again.MusicFiles != 0 {
t.Errorf("a second run sent %d files, want none", again.MusicFiles)
}
// A scan that touched the track but found the same file sends nothing.
mustExec(t, pool, `UPDATE tracks SET updated_at = now()`)
if touched := runOnce(t, svc); touched.MusicFiles != 0 {
t.Errorf("after a scan touched the tracks the run sent %d files, want none", touched.MusicFiles)
}
if p := status(t, svc).Music; p == nil || p.SentFiles != 2 {
t.Errorf("progress after the scan = %+v, want 2 sent", p)
}
// A changed file goes again, to the same place.
path := filepath.Join(lib, "A", "one.flac")
if err := os.WriteFile(path, []byte("one, retagged"), 0o644); err != nil {
t.Fatal(err)
}
later := time.Now().Add(time.Minute)
if err := os.Chtimes(path, later, later); err != nil {
t.Fatal(err)
}
mustExec(t, pool, `UPDATE tracks SET file_size = $1 WHERE file_path = $2`, len("one, retagged"), path)
if changed := runOnce(t, svc); changed.MusicFiles != 1 {
t.Errorf("after one file changed the run sent %d files, want 1", changed.MusicFiles)
}
if got, _ := os.ReadFile(filepath.Join(dest, "music", "A", "one.flac")); string(got) != "one, retagged" {
t.Errorf("music/A/one.flac = %q, want the changed file", got)
}
if entries, _ := os.ReadDir(filepath.Join(dest, "music", "A")); len(entries) != 1 {
t.Errorf("music/A holds %d entries, want just the file", len(entries))
}
}
// lockedBucket is a fakeBucket that refuses to overwrite an object, like a
// bucket with object lock. With all set it refuses every music object, like
// a key that may write only under database/.
type lockedBucket struct {
*fakeBucket
all *atomic.Bool
}
func (b lockedBucket) ServeHTTP(w http.ResponseWriter, r *http.Request) {
if r.Method == http.MethodPut && strings.HasPrefix(r.URL.Path, "/minstrel/music/") {
b.mu.Lock()
_, held := b.puts[r.URL.Path]
b.mu.Unlock()
if held || (b.all != nil && b.all.Load()) {
w.WriteHeader(http.StatusForbidden)
_, _ = w.Write([]byte(`<Error><Code>AccessDenied</Code><Message>object is locked</Message></Error>`))
return
}
}
b.fakeBucket.ServeHTTP(w, r)
}
func TestTheMusicGoesToTheBucketAndARefusedOverwriteIsKept_Integration(t *testing.T) {
svc, pool := newService(t)
ctx := context.Background()
fake := &fakeBucket{puts: map[string][]byte{}}
srv := httptest.NewServer(lockedBucket{fakeBucket: fake})
defer srv.Close()
lib := library(t, pool, map[string]string{"A/one.flac": "one", "B/two.flac": "second"})
svc.Roots = func() []string { return []string{lib} }
u := bucketUpdate(srv.URL)
u.IncludeMusic = true
if err := svc.Save(ctx, u); err != nil {
t.Fatal(err)
}
if first := runOnce(t, svc); first.MusicFiles != 2 {
t.Fatalf("the first run sent %d files, want 2", first.MusicFiles)
}
fake.mu.Lock()
body := string(fake.puts["/minstrel/music/A/one.flac"])
fake.mu.Unlock()
if body != "one" {
t.Errorf("music/A/one.flac in the bucket = %q", body)
}
path := filepath.Join(lib, "A", "one.flac")
if err := os.WriteFile(path, []byte("one, retagged"), 0o644); err != nil {
t.Fatal(err)
}
mustExec(t, pool, `UPDATE tracks SET file_size = $1, updated_at = now() WHERE file_path = $2`, len("one, retagged"), path)
run := runOnce(t, svc)
if run.MusicFiles != 0 || run.MusicNote != "" {
t.Errorf("a refused overwrite: sent %d, note %q; want nothing sent and no note", run.MusicFiles, run.MusicNote)
}
p := status(t, svc).Music
if p == nil || p.SentFiles != 1 || p.RefusedFiles != 1 || len(p.Refused) != 1 ||
p.Refused[0].Path != path || !strings.Contains(p.Refused[0].Error, "AccessDenied") {
t.Errorf("progress = %+v, want one sent and the overwrite refused", p)
}
// Refused is settled until the file changes again.
if again := runOnce(t, svc); again.MusicFiles != 0 {
t.Errorf("the run after a refusal sent %d files, want none", again.MusicFiles)
}
if p := status(t, svc).Music; p == nil || p.RefusedFiles != 1 {
t.Errorf("progress after the next run = %+v, want the refusal kept", p)
}
}
func TestAKeyThatMayNotWriteTheMusicStopsThePassAndASaveTriesAgain_Integration(t *testing.T) {
svc, pool := newService(t)
ctx := context.Background()
fake := &fakeBucket{puts: map[string][]byte{}}
all := &atomic.Bool{}
all.Store(true)
srv := httptest.NewServer(lockedBucket{fakeBucket: fake, all: all})
defer srv.Close()
lib := library(t, pool, map[string]string{"a.flac": "a", "b.flac": "b", "c.flac": "c", "d.flac": "d"})
svc.Roots = func() []string { return []string{lib} }
u := bucketUpdate(srv.URL)
u.IncludeMusic = true
if err := svc.Save(ctx, u); err != nil {
t.Fatal(err)
}
run := runOnce(t, svc)
if run.MusicFiles != 0 || !strings.Contains(run.MusicNote, "refused 3 files in a row") {
t.Errorf("sent %d, note %q; want the pass stopped after three refusals", run.MusicFiles, run.MusicNote)
}
if p := status(t, svc).Music; p == nil || p.RefusedFiles != 3 {
t.Errorf("progress = %+v, want three refused and the fourth not tried", p)
}
// The key is fixed and the settings saved: the refused files go again.
all.Store(false)
if err := svc.Save(ctx, u); err != nil {
t.Fatal(err)
}
if again := runOnce(t, svc); again.MusicFiles != 4 || again.MusicNote != "" {
t.Errorf("after the save: sent %d, note %q; want all four", again.MusicFiles, again.MusicNote)
}
}
+60
View File
@@ -0,0 +1,60 @@
package backup
import (
"errors"
"fmt"
"net/http"
"testing"
)
func TestCommonRootIsTheFolderHoldingEveryLibraryRoot(t *testing.T) {
for _, c := range []struct {
roots []string
want string
}{
{nil, ""},
{[]string{"/music"}, "/music"},
{[]string{"/music/"}, "/music"},
{[]string{"/data/music/a", "/data/music/b"}, "/data/music"},
{[]string{"/data/music", "/data/musicals"}, "/data"},
{[]string{"/music", "/audiobooks"}, "/"},
} {
if got := commonRoot(c.roots); got != c.want {
t.Errorf("commonRoot(%q) = %q, want %q", c.roots, got, c.want)
}
}
}
func TestMusicPathIsTheFileUnderTheCommonRoot(t *testing.T) {
for _, c := range []struct{ root, file, want string }{
{"/data/music", "/data/music/a/Artist/01 Song.flac", "music/a/Artist/01 Song.flac"},
{"/", "/music/x.mp3", "music/music/x.mp3"},
{"/data/music", "/elsewhere/x.mp3", "music/elsewhere/x.mp3"},
{"", "/music/x.mp3", "music/music/x.mp3"},
} {
if got := musicPath(c.root, c.file); got != c.want {
t.Errorf("musicPath(%q, %q) = %q, want %q", c.root, c.file, got, c.want)
}
}
}
func TestObjectRefusedIsOneObjectNotTheWholeBucket(t *testing.T) {
for _, c := range []struct {
err error
want bool
}{
{&Refusal{Status: http.StatusForbidden, Code: "AccessDenied"}, true},
{&Refusal{Status: http.StatusConflict, Code: "ObjectLocked"}, true},
{fmt.Errorf("put: %w", &Refusal{Status: http.StatusBadRequest, Code: "InvalidObjectName"}), true},
{&Refusal{Status: http.StatusForbidden, Code: "SignatureDoesNotMatch"}, false},
{&Refusal{Status: http.StatusNotFound, Code: "NoSuchBucket"}, false},
{&Refusal{Status: http.StatusTooManyRequests}, false},
{&Refusal{Status: http.StatusServiceUnavailable, Code: "SlowDown"}, false},
{&Refusal{Status: http.StatusInternalServerError}, false},
{errors.New("connection refused"), false},
} {
if got := ObjectRefused(c.err); got != c.want {
t.Errorf("ObjectRefused(%v %+v) = %v, want %v", c.err, c.err, got, c.want)
}
}
}
+201
View File
@@ -0,0 +1,201 @@
package backup
import (
"context"
"encoding/json"
"errors"
"fmt"
"slices"
"strings"
"github.com/jackc/pgx/v5"
"github.com/jackc/pgx/v5/pgxpool"
)
var (
// ErrNotEmpty is a restore into an instance that already has accounts.
ErrNotEmpty = errors.New("backup: this Minstrel already has accounts; a backup restores into a new one")
// ErrNewerSchema is a backup written by a newer Minstrel than this one.
ErrNewerSchema = errors.New("backup: the backup was written by a newer version of Minstrel; upgrade this one first")
// ErrNotABackup is contents that aren't rows a backup writes.
ErrNotABackup = errors.New("backup: not a Minstrel backup")
)
// batchRows is how many rows one INSERT carries.
const batchRows = 500
// Restore loads a backup into an empty instance, in one transaction: either
// all of it lands or none of it does.
//
// The instance must have no accounts. Its rows in the tables the backup holds
// are replaced: a fresh instance's seeded settings, and the library its
// startup scan found before anyone signed up, which the scan after the
// restore finds again under the backup's ids. A table the backup doesn't
// hold, because it was written before that table existed, keeps what the
// instance has.
//
// A row's columns are those the backup and the table both have: an older
// backup's rows take the defaults of columns added since, and a column dropped
// since is left out.
func Restore(ctx context.Context, pool *pgxpool.Pool, c Contents) error {
tx, err := pool.Begin(ctx)
if err != nil {
return fmt.Errorf("backup: begin: %w", err)
}
defer func() { _ = tx.Rollback(ctx) }()
schema, err := schemaVersion(ctx, tx)
if err != nil {
return err
}
if c.Schema > schema {
return ErrNewerSchema
}
// Taken before the emptiness check, so a first account made at the same
// moment waits for the restore, then finds the data. A scan still running
// waits too, rather than adding to the library while it is replaced.
if _, err := tx.Exec(ctx, "LOCK TABLE users, artists, albums, tracks IN EXCLUSIVE MODE"); err != nil {
return fmt.Errorf("backup: lock: %w", err)
}
var occupied bool
if err := tx.QueryRow(ctx, "SELECT EXISTS (SELECT 1 FROM users)").Scan(&occupied); err != nil {
return fmt.Errorf("backup: check the instance is new: %w", err)
}
if occupied {
return ErrNotEmpty
}
// Children first, so no row is left pointing at one already gone.
for _, t := range slices.Backward(Tables) {
if _, ok := c.Tables[t.Name]; !ok {
continue
}
if _, err := tx.Exec(ctx, "DELETE FROM "+pgx.Identifier{t.Name}.Sanitize()); err != nil {
return fmt.Errorf("backup: clear %s: %w", t.Name, err)
}
}
for _, t := range Tables {
rows, ok := c.Tables[t.Name]
if !ok {
continue
}
if err := restoreTable(ctx, tx, t.Name, rows); err != nil {
return err
}
}
if err := tx.Commit(ctx); err != nil {
return fmt.Errorf("backup: commit: %w", err)
}
return nil
}
type column struct {
name string
identity bool
sequence bool
}
func restoreTable(ctx context.Context, tx pgx.Tx, table string, rows [][]byte) error {
cols, err := writableColumns(ctx, tx, table)
if err != nil {
return err
}
ident := pgx.Identifier{table}.Sanitize()
overriding := ""
if slices.ContainsFunc(cols, func(c column) bool { return c.identity }) {
overriding = " OVERRIDING SYSTEM VALUE"
}
// Rows go in batches that share one key set; a backup's rows for a table
// all do, since one read wrote them.
var (
batch []json.RawMessage
list string
)
flush := func() error {
if len(batch) == 0 {
return nil
}
payload, err := json.Marshal(batch)
if err != nil {
return err
}
sql := fmt.Sprintf("INSERT INTO %s (%s)%s SELECT %s FROM jsonb_populate_recordset(NULL::%s, $1::jsonb)",
ident, list, overriding, list, ident)
if _, err := tx.Exec(ctx, sql, string(payload)); err != nil {
return fmt.Errorf("backup: restore %s: %w", table, err)
}
batch = batch[:0]
return nil
}
for i, raw := range rows {
var keys map[string]json.RawMessage
if err := json.Unmarshal(raw, &keys); err != nil {
return fmt.Errorf("%w: row %d of %s is not an object", ErrNotABackup, i+1, table)
}
var names []string
for _, c := range cols {
if _, ok := keys[c.name]; ok {
names = append(names, pgx.Identifier{c.name}.Sanitize())
}
}
if len(names) == 0 {
continue
}
slices.Sort(names)
rowList := strings.Join(names, ", ")
if rowList != list || len(batch) == batchRows {
if err := flush(); err != nil {
return err
}
list = rowList
}
batch = append(batch, raw)
}
if err := flush(); err != nil {
return err
}
// A sequence carries on after the highest restored value, so the next row
// doesn't collide with one from the backup. It is left as having issued
// that value, so the sequence itself says how far the restored history
// goes: the library sync compares an app's cursor with it.
for _, c := range cols {
if !c.sequence {
continue
}
col := pgx.Identifier{c.name}.Sanitize()
if _, err := tx.Exec(ctx, fmt.Sprintf(`
SELECT CASE WHEN m > 0 THEN setval(pg_get_serial_sequence($1, $2), m, true)
ELSE setval(pg_get_serial_sequence($1, $2), 1, false) END
FROM (SELECT COALESCE(max(%s), 0) AS m FROM %s) AS top`, col, ident),
table, c.name); err != nil {
return fmt.Errorf("backup: carry on %s's sequence: %w", table, err)
}
}
return nil
}
// writableColumns returns the columns of a table an INSERT can set: all but
// generated ones, which Postgres computes.
func writableColumns(ctx context.Context, tx pgx.Tx, table string) ([]column, error) {
rows, err := tx.Query(ctx, `
SELECT column_name, is_identity = 'YES', pg_get_serial_sequence($1::text, column_name) IS NOT NULL
FROM information_schema.columns
WHERE table_schema = current_schema() AND table_name = $1::text AND is_generated = 'NEVER'
ORDER BY ordinal_position`, table)
if err != nil {
return nil, fmt.Errorf("backup: read %s's columns: %w", table, err)
}
cols, err := pgx.CollectRows(rows, func(row pgx.CollectableRow) (column, error) {
var c column
err := row.Scan(&c.name, &c.identity, &c.sequence)
return c, err
})
if err != nil {
return nil, fmt.Errorf("backup: read %s's columns: %w", table, err)
}
return cols, nil
}
+231
View File
@@ -0,0 +1,231 @@
package backup
import (
"context"
"encoding/xml"
"errors"
"fmt"
"io"
"net/http"
"net/url"
"strconv"
"strings"
"time"
)
// ErrS3 is a request the bucket refused or never answered.
var ErrS3 = errors.New("backup: the S3 bucket refused the request")
// Bucket is a path-style S3 bucket: an endpoint such as Fabled Archives'
// backup port or MinIO, a bucket name, and a key pair. A backup only ever
// adds objects and lists them, so an append-only key is enough.
type Bucket struct {
Endpoint string
Name string
AccessKey string
SecretKey string
// Region defaults to us-east-1, which S3-compatible servers accept.
Region string
Client *http.Client
// now is the signing clock; tests pin it.
now func() time.Time
}
// Object is one stored object.
type Object struct {
Key string
Size int64
LastModified time.Time
}
func (b Bucket) region() string {
if b.Region == "" {
return "us-east-1"
}
return b.Region
}
func (b Bucket) client() *http.Client {
if b.Client != nil {
return b.Client
}
return http.DefaultClient
}
func (b Bucket) clock() time.Time {
if b.now != nil {
return b.now()
}
return time.Now()
}
// objectURL is the path-style URL of key, encoded exactly as it is signed.
func (b Bucket) objectURL(key string) (*url.URL, error) {
u, err := url.Parse(strings.TrimRight(b.Endpoint, "/"))
if err != nil || (u.Scheme != "http" && u.Scheme != "https") || u.Host == "" {
return nil, fmt.Errorf("%w: the endpoint %q is not an http(s) address", ErrS3, b.Endpoint)
}
if b.Name == "" {
return nil, fmt.Errorf("%w: no bucket name", ErrS3)
}
raw := "/" + uriEncode(b.Name)
if key != "" {
for s := range strings.SplitSeq(key, "/") {
raw += "/" + uriEncode(s)
}
}
u.RawPath = u.EscapedPath() + raw
u.Path, _ = url.PathUnescape(u.RawPath)
return u, nil
}
func (b Bucket) do(req *http.Request, payloadSHA256 string) (*http.Response, error) {
sign(req, payloadSHA256, b.AccessKey, b.SecretKey, b.region(), b.clock())
resp, err := b.client().Do(req)
if err != nil {
return nil, fmt.Errorf("%w: %s is unreachable: %v", ErrS3, req.URL.Host, err)
}
if resp.StatusCode/100 != 2 {
defer func() { _ = resp.Body.Close() }()
return nil, refusal(resp)
}
return resp, nil
}
// Put stores body under key. size is the body's length and payloadSHA256
// its hex SHA-256, which the request is signed over.
func (b Bucket) Put(ctx context.Context, key string, body io.Reader, size int64, payloadSHA256 string) error {
u, err := b.objectURL(key)
if err != nil {
return err
}
req, err := http.NewRequestWithContext(ctx, http.MethodPut, u.String(), body)
if err != nil {
return err
}
req.URL = u
req.ContentLength = size
if size == 0 {
req.Body = http.NoBody
}
resp, err := b.do(req, payloadSHA256)
if err != nil {
return err
}
drain(resp)
return nil
}
// List returns up to limit objects whose keys start with prefix, in key
// order.
func (b Bucket) List(ctx context.Context, prefix string, limit int) ([]Object, error) {
if limit < 1 {
limit = 1000
}
var out []Object
token := ""
for {
u, err := b.objectURL("")
if err != nil {
return nil, err
}
q := url.Values{"list-type": {"2"}, "prefix": {prefix}}
if page := limit - len(out); page < 1000 {
q.Set("max-keys", strconv.Itoa(page))
}
if token != "" {
q.Set("continuation-token", token)
}
u.RawQuery = canonicalQuery(q)
req, err := http.NewRequestWithContext(ctx, http.MethodGet, u.String(), nil)
if err != nil {
return nil, err
}
req.URL = u
resp, err := b.do(req, emptySHA256)
if err != nil {
return nil, err
}
var page struct {
Contents []struct {
Key string `xml:"Key"`
Size int64 `xml:"Size"`
LastModified string `xml:"LastModified"`
} `xml:"Contents"`
IsTruncated bool `xml:"IsTruncated"`
NextContinuationToken string `xml:"NextContinuationToken"`
}
err = xml.NewDecoder(io.LimitReader(resp.Body, 16<<20)).Decode(&page)
drain(resp)
if err != nil {
return nil, fmt.Errorf("%w: the listing is not S3 XML: %v", ErrS3, err)
}
for _, c := range page.Contents {
modified, _ := time.Parse(time.RFC3339, c.LastModified)
out = append(out, Object{Key: c.Key, Size: c.Size, LastModified: modified})
}
if !page.IsTruncated || page.NextContinuationToken == "" || len(out) >= limit {
return out, nil
}
token = page.NextContinuationToken
}
}
// Refusal is an S3 error response: its HTTP status, S3's code, and the
// reason in words. It matches ErrS3.
type Refusal struct {
Status int
Code string
text string
}
func (r *Refusal) Error() string { return r.text }
func (r *Refusal) Unwrap() error { return ErrS3 }
// refusal turns an S3 error response into words an admin can act on.
func refusal(resp *http.Response) error {
var e struct {
Code string `xml:"Code"`
Message string `xml:"Message"`
}
_ = xml.NewDecoder(io.LimitReader(resp.Body, 64<<10)).Decode(&e)
hint := map[string]string{
"SignatureDoesNotMatch": "the secret key is wrong",
"InvalidAccessKeyId": "the access key is not known to this server",
"AccessDenied": "the key may not do this in this bucket",
"NoSuchBucket": "there is no bucket by that name",
"SlowDown": "the server asks to slow down",
}[e.Code]
r := &Refusal{Status: resp.StatusCode, Code: e.Code}
switch {
case hint != "":
r.text = fmt.Sprintf("%v: HTTP %d %s: %s", ErrS3, resp.StatusCode, e.Code, hint)
case e.Code != "":
r.text = fmt.Sprintf("%v: HTTP %d %s: %s", ErrS3, resp.StatusCode, e.Code, e.Message)
default:
r.text = fmt.Sprintf("%v: HTTP %d", ErrS3, resp.StatusCode)
}
return r
}
// ObjectRefused reports whether err is the bucket refusing one object (an
// overwrite a write-once target won't take, a name it won't store) rather
// than refusing everything: bad keys, a missing bucket, a request to slow
// down, or a server error all stop a run instead.
func ObjectRefused(err error) bool {
var r *Refusal
if !errors.As(err, &r) || r.Status/100 != 4 || r.Status == http.StatusTooManyRequests {
return false
}
switch r.Code {
case "SignatureDoesNotMatch", "InvalidAccessKeyId", "NoSuchBucket", "SlowDown",
"RequestTimeTooSkewed", "ExpiredToken", "InvalidToken":
return false
}
return true
}
func drain(resp *http.Response) {
_, _ = io.Copy(io.Discard, io.LimitReader(resp.Body, 1<<20))
_ = resp.Body.Close()
}
+154
View File
@@ -0,0 +1,154 @@
package backup
import (
"context"
"errors"
"fmt"
"io"
"net/http"
"net/http/httptest"
"strings"
"testing"
"time"
)
// AWS's published examples ("Examples: Signature Calculations in AWS
// Signature Version 4", Amazon S3 API reference).
const (
awsAccessKey = "AKIAIOSFODNN7EXAMPLE"
awsSecretKey = "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY"
awsCred = "AWS4-HMAC-SHA256 Credential=AKIAIOSFODNN7EXAMPLE/20130524/us-east-1/s3/aws4_request"
)
var awsTime = time.Date(2013, 5, 24, 0, 0, 0, 0, time.UTC)
func TestSignMatchesAWSsGetObjectExample(t *testing.T) {
req, _ := http.NewRequest(http.MethodGet, "http://examplebucket.s3.amazonaws.com/test.txt", nil)
req.Header.Set("Range", "bytes=0-9")
sign(req, emptySHA256, awsAccessKey, awsSecretKey, "us-east-1", awsTime)
want := awsCred + ", SignedHeaders=host;range;x-amz-content-sha256;x-amz-date, Signature=f0e8bdb87c964420e857bd35b5d6ed310bd44f0170aba48dd91039c6036bdb41"
if got := req.Header.Get("Authorization"); got != want {
t.Errorf("Authorization =\n%s\nwant\n%s", got, want)
}
}
func TestSignMatchesAWSsPutObjectExample(t *testing.T) {
req, _ := http.NewRequest(http.MethodPut, "http://examplebucket.s3.amazonaws.com/test$file.text", strings.NewReader("Welcome to Amazon S3."))
req.Header.Set("Date", "Fri, 24 May 2013 00:00:00 GMT")
req.Header.Set("X-Amz-Storage-Class", "REDUCED_REDUNDANCY")
payload := hexSHA256([]byte("Welcome to Amazon S3."))
if payload != "44ce7dd67c959e0d3524ffac1771dfbba87d2b6b4b4e99e42034a8b803f8b072" {
t.Fatalf("payload hash = %s", payload)
}
sign(req, payload, awsAccessKey, awsSecretKey, "us-east-1", awsTime)
want := awsCred + ", SignedHeaders=date;host;x-amz-content-sha256;x-amz-date;x-amz-storage-class, Signature=98ad721746da40c64f1a55b78f14c238d841ea1380cd77a1b5971af0ece108bd"
if got := req.Header.Get("Authorization"); got != want {
t.Errorf("Authorization =\n%s\nwant\n%s", got, want)
}
}
func TestSignMatchesAWSsListObjectsExample(t *testing.T) {
req, _ := http.NewRequest(http.MethodGet, "http://examplebucket.s3.amazonaws.com/?max-keys=2&prefix=J", nil)
sign(req, emptySHA256, awsAccessKey, awsSecretKey, "us-east-1", awsTime)
want := awsCred + ", SignedHeaders=host;x-amz-content-sha256;x-amz-date, Signature=34b48302e7b5fa45bde8084f4b7868a86f0a534bc59db6670ed5711ef69dc6f7"
if got := req.Header.Get("Authorization"); got != want {
t.Errorf("Authorization =\n%s\nwant\n%s", got, want)
}
}
func TestUriEncodeKeepsOnlyUnreservedCharacters(t *testing.T) {
for in, want := range map[string]string{
"01 - Song.flac": "01%20-%20Song.flac",
"a+b=c&d": "a%2Bb%3Dc%26d",
"Björk~_.": "Bj%C3%B6rk~_.",
"a/b": "a%2Fb",
} {
if got := uriEncode(in); got != want {
t.Errorf("uriEncode(%q) = %q, want %q", in, got, want)
}
}
}
func testBucket(url string) Bucket {
return Bucket{Endpoint: url, Name: "minstrel", AccessKey: "ak", SecretKey: "sk",
now: func() time.Time { return awsTime }}
}
func TestPutSendsTheBodyToItsPathStyleKey(t *testing.T) {
var gotPath, gotSHA, gotBody, gotAuth string
var gotLen int64
srv := httptest.NewServer(http.HandlerFunc(func(_ http.ResponseWriter, r *http.Request) {
gotPath, gotSHA, gotAuth, gotLen = r.URL.EscapedPath(), r.Header.Get("X-Amz-Content-Sha256"), r.Header.Get("Authorization"), r.ContentLength
b, _ := io.ReadAll(r.Body)
gotBody = string(b)
}))
defer srv.Close()
body := "flac bytes"
sum := hexSHA256([]byte(body))
if err := testBucket(srv.URL).Put(context.Background(), "music/A B/01 + x.flac", strings.NewReader(body), int64(len(body)), sum); err != nil {
t.Fatal(err)
}
if gotPath != "/minstrel/music/A%20B/01%20%2B%20x.flac" {
t.Errorf("path = %s", gotPath)
}
if gotSHA != sum || gotBody != body || gotLen != int64(len(body)) {
t.Errorf("sha %s body %q length %d", gotSHA, gotBody, gotLen)
}
if !strings.HasPrefix(gotAuth, "AWS4-HMAC-SHA256 Credential=ak/20130524/us-east-1/s3/aws4_request") {
t.Errorf("Authorization = %s", gotAuth)
}
}
func TestListFollowsContinuationTokens(t *testing.T) {
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
q := r.URL.Query()
if r.URL.Path != "/minstrel" || q.Get("list-type") != "2" || q.Get("prefix") != "database/" {
http.Error(w, "unexpected "+r.URL.String(), http.StatusBadRequest)
return
}
if q.Get("continuation-token") == "" {
_, _ = fmt.Fprint(w, `<ListBucketResult><IsTruncated>true</IsTruncated><NextContinuationToken>next</NextContinuationToken>
<Contents><Key>database/minstrel-20261101T030000Z.tar</Key><Size>10</Size><LastModified>2026-11-01T03:00:05.000Z</LastModified></Contents>
<Contents><Key>database/minstrel-20261102T030000Z.tar</Key><Size>11</Size><LastModified>2026-11-02T03:00:05.000Z</LastModified></Contents>
</ListBucketResult>`)
return
}
_, _ = fmt.Fprint(w, `<ListBucketResult><IsTruncated>false</IsTruncated>
<Contents><Key>database/minstrel-20261103T030000Z.tar</Key><Size>12</Size><LastModified>2026-11-03T03:00:05.000Z</LastModified></Contents>
</ListBucketResult>`)
}))
defer srv.Close()
objs, err := testBucket(srv.URL).List(context.Background(), "database/", 0)
if err != nil {
t.Fatal(err)
}
if len(objs) != 3 || objs[2].Key != "database/minstrel-20261103T030000Z.tar" || objs[1].Size != 11 ||
!objs[0].LastModified.Equal(time.Date(2026, 11, 1, 3, 0, 5, 0, time.UTC)) {
t.Errorf("objects = %+v", objs)
}
}
func TestARefusalSaysWhatIsWrong(t *testing.T) {
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
w.WriteHeader(http.StatusForbidden)
_, _ = fmt.Fprint(w, `<Error><Code>SignatureDoesNotMatch</Code><Message>The request signature we calculated does not match</Message></Error>`)
}))
defer srv.Close()
_, err := testBucket(srv.URL).List(context.Background(), "", 1)
if !errors.Is(err, ErrS3) || !strings.Contains(err.Error(), "HTTP 403 SignatureDoesNotMatch: the secret key is wrong") {
t.Errorf("err = %v", err)
}
}
func TestAnUnreachableOrMalformedEndpointIsAnS3Error(t *testing.T) {
srv := httptest.NewServer(http.NotFoundHandler())
srv.Close()
for name, endpoint := range map[string]string{"closed": srv.URL, "not http": "ftp://archives", "empty": ""} {
if _, err := testBucket(endpoint).List(context.Background(), "", 1); !errors.Is(err, ErrS3) {
t.Errorf("%s: err = %v, want ErrS3", name, err)
}
}
}
+755
View File
@@ -0,0 +1,755 @@
package backup
import (
"context"
"crypto/sha256"
"encoding/hex"
"errors"
"fmt"
"io"
"log/slog"
"net/url"
"os"
"path"
"path/filepath"
"strings"
"sync"
"time"
"github.com/jackc/pgx/v5/pgtype"
"github.com/jackc/pgx/v5/pgxpool"
"git.fabledsword.com/bvandeusen/minstrel/internal/db/dbq"
"git.fabledsword.com/bvandeusen/minstrel/internal/secret"
)
// Where a backup can go.
const (
DestinationFolder = "folder"
DestinationS3 = "s3"
)
// What started a run.
const (
TriggerSchedule = "schedule"
TriggerManual = "manual"
)
// The verdict a Status gives, worst first in the order an admin acts on them.
const (
StateOff = "off" // the nightly backup is switched off
StateIncomplete = "incomplete" // switched on, but the destination isn't set up
StateFailing = "failing" // the latest run failed
StateWaiting = "waiting" // set up, and no backup has finished yet
StateOverdue = "overdue" // the latest good backup is more than a day old
StateOK = "ok"
)
const (
// RunLimit bounds one run, snapshot and upload together. The web panel
// stops polling a run a little after this, since the run is over by then.
RunLimit = 30 * time.Minute
// A nightly backup that hasn't succeeded for this long is overdue; the
// slack past a day covers a run that starts late or takes a while.
overdueAfter = 26 * time.Hour
tickEvery = 5 * time.Minute
// A failed run is tried again after at least this long, and after longer
// each time it fails again, so a destination that is down isn't hammered.
retryAfter = time.Hour
recentRuns = 10
// Run history older than this is pruned after each run.
keepRunsFor = 90 * 24 * time.Hour
testTimeout = 30 * time.Second
// The database backups' folder inside the destination. The music, when
// it is backed up, sits beside it in the same destination.
databaseDir = "database"
)
var (
// ErrRunning is returned by Start while a backup is running.
ErrRunning = errors.New("backup: a backup is already running")
// ErrNotReady is returned by Start when the destination isn't set up.
ErrNotReady = errors.New("backup: the destination isn't set up yet")
// ErrClosed is returned by Start once the server is shutting down.
ErrClosed = errors.New("backup: the server is shutting down")
)
// FieldError is a setting that can't be saved, with the reason in words.
type FieldError struct {
Field string `json:"field"`
Message string `json:"message"`
}
func (e *FieldError) Error() string { return e.Field + ": " + e.Message }
// Config is the backup's settings as an admin sees them. The bucket's secret
// key is never sent back, only whether one is stored.
type Config struct {
Enabled bool `json:"enabled"`
Destination string `json:"destination"`
Folder string `json:"folder"`
Endpoint string `json:"endpoint"`
Bucket string `json:"bucket"`
AccessKey string `json:"access_key"`
HasSecretKey bool `json:"has_secret_key"`
RunHour int `json:"run_hour"`
UploadLimitKiB int `json:"upload_limit_kib"`
IncludeMusic bool `json:"include_music"`
}
// Update is a save from the admin card. An empty SecretKey keeps the stored
// one, so the key doesn't have to be typed again on every save.
type Update struct {
Enabled bool `json:"enabled"`
Destination string `json:"destination"`
Folder string `json:"folder"`
Endpoint string `json:"endpoint"`
Bucket string `json:"bucket"`
AccessKey string `json:"access_key"`
SecretKey string `json:"secret_key"`
RunHour int `json:"run_hour"`
UploadLimitKiB int `json:"upload_limit_kib"`
IncludeMusic bool `json:"include_music"`
}
// Run is one attempt at a backup.
type Run struct {
ID string `json:"id"`
Trigger string `json:"trigger"`
Status string `json:"status"`
StartedAt time.Time `json:"started_at"`
FinishedAt *time.Time `json:"finished_at"`
Location string `json:"location"`
Bytes int64 `json:"bytes"`
Error string `json:"error"`
// What the run sent of the music, and why that stopped early, if it did.
MusicFiles int `json:"music_files"`
MusicBytes int64 `json:"music_bytes"`
MusicNote string `json:"music_note"`
}
// Status is everything the admin panel shows: a verdict, what stands in its
// way, and the recent runs.
type Status struct {
Config Config `json:"config"`
State string `json:"state"`
Problems []string `json:"problems"`
RunningSince *time.Time `json:"running_since"`
LastSuccess *time.Time `json:"last_success"`
FailingSince *time.Time `json:"failing_since"`
// NextRun is when the schedule starts the next backup, or nil when it
// won't: switched off, or not set up.
NextRun *time.Time `json:"next_run"`
RunLimitSeconds int `json:"run_limit_seconds"`
Runs []Run `json:"runs"`
// Music is how much of the library the backup holds, when it includes
// the music.
Music *MusicProgress `json:"music"`
}
// TestResult is what a destination test found: how many backups are there
// already and when the newest was made.
type TestResult struct {
Backups int `json:"backups"`
Latest *time.Time `json:"latest"`
}
// Service runs Minstrel's own backup: nightly when switched on, or when an
// admin asks. One run at a time; each is recorded in backup_runs.
type Service struct {
pool *pgxpool.Pool
logger *slog.Logger
build string
now func() time.Time
// Roots lists the library's folders. The music's paths in the backup are
// relative to the folder holding them all.
Roots func() []string
base context.Context
cancel context.CancelFunc
wg sync.WaitGroup
mu sync.Mutex
runningSince time.Time
closed bool
}
// NewService returns a Service over pool. build is the server's version,
// written into each backup's manifest.
func NewService(pool *pgxpool.Pool, logger *slog.Logger, build string) *Service {
base, cancel := context.WithCancel(context.Background())
return &Service{pool: pool, logger: logger, build: build, now: time.Now, base: base, cancel: cancel}
}
func (s *Service) settings(ctx context.Context) (dbq.BackupSetting, error) {
st, err := dbq.New(s.pool).GetBackupSettings(ctx)
if err != nil {
return st, fmt.Errorf("backup: read settings: %w", err)
}
return st, nil
}
func view(st dbq.BackupSetting) Config {
return Config{
Enabled: st.Enabled,
Destination: st.Destination,
Folder: st.Folder,
Endpoint: st.Endpoint,
Bucket: st.Bucket,
AccessKey: st.AccessKey,
HasSecretKey: st.SecretKey != "",
RunHour: int(st.RunHour),
UploadLimitKiB: int(st.UploadLimitKib),
IncludeMusic: st.IncludeMusic,
}
}
// Config returns the settings as an admin sees them.
func (s *Service) Config(ctx context.Context) (Config, error) {
st, err := s.settings(ctx)
return view(st), err
}
// merge applies u over the stored settings: it checks each field, and seals
// a new secret key. It returns a *FieldError for a value that can't be kept.
func merge(st dbq.BackupSetting, u Update) (dbq.BackupSetting, error) {
if u.Destination != DestinationFolder && u.Destination != DestinationS3 {
return st, &FieldError{"destination", "Choose a folder or an S3 bucket."}
}
if u.RunHour < 0 || u.RunHour > 23 {
return st, &FieldError{"run_hour", "The hour is from 0 to 23."}
}
if u.UploadLimitKiB < 0 || u.UploadLimitKiB > 1<<30 {
return st, &FieldError{"upload_limit_kib", "The upload limit is 0 (no limit) or more."}
}
folder := strings.TrimSpace(u.Folder)
if folder != "" && !filepath.IsAbs(folder) {
return st, &FieldError{"folder", "Give the folder's full path, starting with /."}
}
endpoint := strings.TrimRight(strings.TrimSpace(u.Endpoint), "/")
if endpoint != "" {
e, err := url.Parse(endpoint)
if err != nil || (e.Scheme != "http" && e.Scheme != "https") || e.Host == "" {
return st, &FieldError{"endpoint", "Give the endpoint as a URL, such as https://s3.example.com."}
}
}
st.Enabled = u.Enabled
st.Destination = u.Destination
st.Folder = filepath.Clean(folder)
if folder == "" {
st.Folder = ""
}
st.Endpoint = endpoint
st.Bucket = strings.TrimSpace(u.Bucket)
st.AccessKey = strings.TrimSpace(u.AccessKey)
st.RunHour = int16(u.RunHour)
st.UploadLimitKib = int32(u.UploadLimitKiB)
st.IncludeMusic = u.IncludeMusic
if u.SecretKey != "" {
sealed, err := secret.Seal(secret.BackupSecretKey.Purpose(), "", u.SecretKey)
if err != nil {
return st, err
}
st.SecretKey = sealed
}
return st, nil
}
// Save stores u. A value that can't be kept is a *FieldError and nothing is
// stored. Settings that are incomplete still save; Status lists what's missing.
func (s *Service) Save(ctx context.Context, u Update) error {
st, err := s.settings(ctx)
if err != nil {
return err
}
st, err = merge(st, u)
if err != nil {
return err
}
return dbq.New(s.pool).SaveBackupSettings(ctx, dbq.SaveBackupSettingsParams{
Enabled: st.Enabled,
Destination: st.Destination,
Folder: st.Folder,
Endpoint: st.Endpoint,
Bucket: st.Bucket,
AccessKey: st.AccessKey,
SecretKey: st.SecretKey,
RunHour: st.RunHour,
UploadLimitKib: st.UploadLimitKib,
IncludeMusic: st.IncludeMusic,
})
}
// problems is what stops a backup from running with these settings, in words.
func problems(st dbq.BackupSetting) []string {
var out []string
switch st.Destination {
case DestinationFolder:
if st.Folder == "" {
out = append(out, "Choose the folder backups are written to.")
}
case DestinationS3:
if st.Endpoint == "" {
out = append(out, "Enter the bucket's endpoint.")
}
if st.Bucket == "" {
out = append(out, "Enter the bucket's name.")
}
if st.AccessKey == "" {
out = append(out, "Enter the access key.")
}
if st.SecretKey == "" {
out = append(out, "Enter the secret key.")
}
default:
out = append(out, "Choose where backups go.")
}
return out
}
func timeOf(t pgtype.Timestamptz) time.Time {
if !t.Valid || t.Time.Unix() <= 0 {
return time.Time{}
}
return t.Time
}
func ptr(t time.Time) *time.Time {
if t.IsZero() {
return nil
}
return &t
}
// runTimes is what the verdict and the schedule turn on. A zero time is a run
// that never happened.
type runTimes struct {
lastSuccess, lastAttempt, failingSince time.Time
}
func (s *Service) times(ctx context.Context) (runTimes, error) {
row, err := dbq.New(s.pool).BackupRunTimes(ctx)
if err != nil {
return runTimes{}, fmt.Errorf("backup: read run times: %w", err)
}
return runTimes{timeOf(row.LastSuccess), timeOf(row.LastAttempt), timeOf(row.FailingSince)}, nil
}
// slotBefore is the latest scheduled start at or before now: today at hour
// UTC, or yesterday's if that hour hasn't come yet.
func slotBefore(now time.Time, hour int) time.Time {
now = now.UTC()
slot := time.Date(now.Year(), now.Month(), now.Day(), hour, 0, 0, 0, time.UTC)
if slot.After(now) {
slot = slot.AddDate(0, 0, -1)
}
return slot
}
// due reports when the schedule next starts a backup: now or earlier means
// the next tick starts one. A slot that passed with no attempt since is due,
// so a server that was down at the hour catches up. A run that failed is
// tried again, waiting at least retryAfter and then as long again as it has
// been failing, until it succeeds or the next slot comes.
func due(now time.Time, hour int, t runTimes) time.Time {
slot := slotBefore(now, hour)
if t.lastAttempt.Before(slot) {
return slot
}
next := slot.AddDate(0, 0, 1)
if !t.failingSince.IsZero() {
wait := max(retryAfter, t.lastAttempt.Sub(t.failingSince))
if retry := t.lastAttempt.Add(wait); retry.Before(next) {
return retry
}
}
return next
}
func verdict(st dbq.BackupSetting, probs []string, t runTimes, now time.Time) string {
switch {
case !st.Enabled:
return StateOff
case len(probs) > 0:
return StateIncomplete
case !t.failingSince.IsZero():
return StateFailing
case t.lastSuccess.IsZero():
return StateWaiting
case now.Sub(t.lastSuccess) > overdueAfter:
return StateOverdue
}
return StateOK
}
func uuidText(u pgtype.UUID) string {
b := u.Bytes
return fmt.Sprintf("%x-%x-%x-%x-%x", b[0:4], b[4:6], b[6:8], b[8:10], b[10:16])
}
// Status is the verdict, what stands in its way, and the recent runs.
func (s *Service) Status(ctx context.Context) (Status, error) {
st, err := s.settings(ctx)
if err != nil {
return Status{}, err
}
t, err := s.times(ctx)
if err != nil {
return Status{}, err
}
rows, err := dbq.New(s.pool).ListBackupRuns(ctx, recentRuns)
if err != nil {
return Status{}, fmt.Errorf("backup: list runs: %w", err)
}
now := s.now()
probs := problems(st)
out := Status{
Config: view(st),
State: verdict(st, probs, t, now),
Problems: probs,
LastSuccess: ptr(t.lastSuccess),
FailingSince: ptr(t.failingSince),
RunLimitSeconds: int(runLimit(st) / time.Second),
Runs: make([]Run, 0, len(rows)),
}
if out.Problems == nil {
out.Problems = []string{}
}
if st.IncludeMusic {
if out.Music, err = s.musicProgress(ctx); err != nil {
return Status{}, err
}
}
s.mu.Lock()
out.RunningSince = ptr(s.runningSince)
s.mu.Unlock()
if st.Enabled && len(probs) == 0 {
next := due(now, int(st.RunHour), t)
if next.Before(now) {
// Due now: the next tick starts it.
next = now
}
out.NextRun = &next
}
for _, r := range rows {
out.Runs = append(out.Runs, Run{
ID: uuidText(r.ID),
Trigger: r.Trigger,
Status: r.Status,
StartedAt: r.StartedAt.Time,
FinishedAt: ptr(timeOf(r.FinishedAt)),
Location: r.Location,
Bytes: r.Bytes,
Error: r.Error,
MusicFiles: int(r.MusicFiles),
MusicBytes: r.MusicBytes,
MusicNote: r.MusicNote,
})
}
return out, nil
}
func bucketOf(st dbq.BackupSetting) (Bucket, error) {
key, err := secret.Open(secret.BackupSecretKey.Purpose(), "", st.SecretKey)
if err != nil {
return Bucket{}, fmt.Errorf("backup: the stored secret key doesn't open; enter it again: %w", err)
}
return Bucket{Endpoint: st.Endpoint, Name: st.Bucket, AccessKey: st.AccessKey, SecretKey: key}, nil
}
// Test checks the destination u describes, falling back to the stored
// secret key when u leaves it empty. It only reads: it lists the backups
// already there and writes nothing.
func (s *Service) Test(ctx context.Context, u Update) (TestResult, error) {
st, err := s.settings(ctx)
if err != nil {
return TestResult{}, err
}
st, err = merge(st, u)
if err != nil {
return TestResult{}, err
}
if probs := problems(st); len(probs) > 0 {
return TestResult{}, fmt.Errorf("%w: %s", ErrNotReady, strings.Join(probs, " "))
}
ctx, cancel := context.WithTimeout(ctx, testTimeout)
defer cancel()
var res TestResult
note := func(name string, at time.Time) {
if !IsFileName(name) {
return
}
res.Backups++
if res.Latest == nil || at.After(*res.Latest) {
res.Latest = &at
}
}
if st.Destination == DestinationFolder {
if err := isFolder(st.Folder); err != nil {
return res, err
}
entries, err := os.ReadDir(filepath.Join(st.Folder, databaseDir))
if err != nil && !errors.Is(err, os.ErrNotExist) {
return res, fmt.Errorf("backup: read %s: %w", filepath.Join(st.Folder, databaseDir), err)
}
for _, e := range entries {
if info, err := e.Info(); err == nil && info.Mode().IsRegular() {
note(e.Name(), info.ModTime())
}
}
return res, nil
}
b, err := bucketOf(st)
if err != nil {
return res, err
}
objs, err := b.List(ctx, databaseDir+"/", 1000)
if err != nil {
return res, err
}
for _, o := range objs {
note(path.Base(o.Key), o.LastModified)
}
return res, nil
}
// isFolder refuses a folder that isn't there. A backup never creates the
// destination folder itself: a missing one usually means a volume that
// isn't mounted, and writing beside it would fill the server's own disk.
func isFolder(dir string) error {
info, err := os.Stat(dir)
if err != nil {
return fmt.Errorf("backup: the folder %s isn't there; is its volume mounted? %w", dir, err)
}
if !info.IsDir() {
return fmt.Errorf("backup: %s is a file, not a folder", dir)
}
return nil
}
// Start begins a backup in the background and returns at once. It returns
// ErrRunning while one runs, ErrNotReady when the destination isn't set up,
// and ErrClosed once the server is shutting down.
func (s *Service) Start(ctx context.Context, trigger string) error {
st, err := s.settings(ctx)
if err != nil {
return err
}
if len(problems(st)) > 0 {
return ErrNotReady
}
s.mu.Lock()
defer s.mu.Unlock()
if s.closed {
return ErrClosed
}
if !s.runningSince.IsZero() {
return ErrRunning
}
s.runningSince = s.now()
s.wg.Add(1)
go func() {
defer s.wg.Done()
defer func() {
s.mu.Lock()
s.runningSince = time.Time{}
s.mu.Unlock()
}()
s.run(trigger, st)
}()
return nil
}
// Running reports when the running backup started, or the zero time.
func (s *Service) Running() time.Time {
s.mu.Lock()
defer s.mu.Unlock()
return s.runningSince
}
// runLimit is the longest a run may take: the database file, and the
// music's nightly budget when the backup includes the music.
func runLimit(st dbq.BackupSetting) time.Duration {
if st.IncludeMusic {
return RunLimit + MusicBudget
}
return RunLimit
}
func (s *Service) run(trigger string, st dbq.BackupSetting) {
ctx, cancel := context.WithTimeout(s.base, RunLimit)
defer cancel()
q := dbq.New(s.pool)
started, err := q.StartBackupRun(ctx, trigger)
if err != nil {
s.logger.Error("backup: record the run's start", "err", err)
return
}
location, size, err := s.write(ctx, st, started.StartedAt.Time)
// The music goes only after the database file has landed, which also
// proves the destination takes writes.
var music musicSent
if err == nil && st.IncludeMusic {
mctx, mcancel := context.WithTimeout(s.base, MusicBudget)
music = s.sendMusic(mctx, st)
mcancel()
}
// The finish is recorded on its own context, so a run stopped by a
// shutdown or its time limit still says how it ended.
fctx, fcancel := context.WithTimeout(context.Background(), 10*time.Second)
defer fcancel()
fin := dbq.FinishBackupRunParams{
ID: started.ID, Status: "succeeded", Location: location, Bytes: size,
MusicFiles: int32(music.files), MusicBytes: music.bytes, MusicNote: music.note,
}
if err != nil {
fin.Status, fin.Error = "failed", describe(err)
s.logger.Warn("backup failed", "trigger", trigger, "err", err)
} else {
s.logger.Info("backup written", "trigger", trigger, "location", location, "bytes", size,
"music_files", music.files, "music_bytes", music.bytes, "music_note", music.note)
}
if err := q.FinishBackupRun(fctx, fin); err != nil {
s.logger.Error("backup: record the run's end", "err", err)
}
if _, err := q.PruneBackupRuns(fctx, pgtype.Timestamptz{Time: s.now().Add(-keepRunsFor), Valid: true}); err != nil {
s.logger.Warn("backup: prune old runs", "err", err)
}
}
// describe is a run's error as the panel shows it.
func describe(err error) string {
switch {
case errors.Is(err, context.DeadlineExceeded):
return fmt.Sprintf("The backup took longer than %d minutes and was stopped.", int(RunLimit/time.Minute))
case errors.Is(err, context.Canceled):
return "The server stopped while the backup was running."
}
return strings.TrimPrefix(err.Error(), "backup: ")
}
// write snapshots the database into one backup file at the destination and
// returns where it went and its size.
func (s *Service) write(ctx context.Context, st dbq.BackupSetting, at time.Time) (string, int64, error) {
if st.Destination == DestinationFolder {
// Checked before the snapshot, so a missing volume costs nothing.
if err := isFolder(st.Folder); err != nil {
return "", 0, err
}
}
c, err := Snapshot(ctx, s.pool)
if err != nil {
return "", 0, err
}
if st.Destination == DestinationFolder {
dir := filepath.Join(st.Folder, databaseDir)
if err := os.Mkdir(dir, 0o700); err != nil && !errors.Is(err, os.ErrExist) {
return "", 0, fmt.Errorf("backup: create %s: %w", dir, err)
}
file, _, err := WriteFile(dir, c, at, s.build)
if err != nil {
return "", 0, err
}
info, err := os.Stat(file)
if err != nil {
return file, 0, err
}
return file, info.Size(), nil
}
b, err := bucketOf(st)
if err != nil {
return "", 0, err
}
// The file is written to a temporary file first: S3 signs over the
// body's hash and needs its length before the upload starts.
tmp, err := os.CreateTemp("", ".minstrel-backup-*.tar")
if err != nil {
return "", 0, fmt.Errorf("backup: create a temporary file: %w", err)
}
defer func() {
_ = tmp.Close()
_ = os.Remove(tmp.Name())
}()
h := sha256.New()
if _, err := Write(io.MultiWriter(tmp, h), c, at, s.build); err != nil {
return "", 0, fmt.Errorf("backup: write the temporary file: %w", err)
}
size, err := tmp.Seek(0, io.SeekCurrent)
if err == nil {
_, err = tmp.Seek(0, io.SeekStart)
}
if err != nil {
return "", 0, fmt.Errorf("backup: rewind the temporary file: %w", err)
}
key := databaseDir + "/" + FileName(at)
body := Throttle(ctx, tmp, int64(st.UploadLimitKib)*1024)
if err := b.Put(ctx, key, body, size, hex.EncodeToString(h.Sum(nil))); err != nil {
return "", 0, err
}
return key, size, nil
}
// Tick starts a scheduled backup when one is due. A backup that is already
// running, or settings that aren't ready, mean there's nothing to do.
func (s *Service) Tick(ctx context.Context) {
st, err := s.settings(ctx)
if err != nil {
s.logger.Warn("backup: tick", "err", err)
return
}
if !st.Enabled || len(problems(st)) > 0 {
return
}
t, err := s.times(ctx)
if err != nil {
s.logger.Warn("backup: tick", "err", err)
return
}
if due(s.now(), int(st.RunHour), t).After(s.now()) {
return
}
if err := s.Start(ctx, TriggerSchedule); err != nil && !errors.Is(err, ErrRunning) && !errors.Is(err, ErrClosed) {
s.logger.Warn("backup: start the scheduled backup", "err", err)
}
}
// Recover marks runs a stopped server left running as failed. It runs once
// at startup, before the first tick.
func (s *Service) Recover(ctx context.Context) {
n, err := dbq.New(s.pool).FailUnfinishedBackupRuns(ctx)
if err != nil {
s.logger.Warn("backup: recover unfinished runs", "err", err)
return
}
if n > 0 {
s.logger.Warn("backup: a run was cut off by the last shutdown", "runs", n)
}
}
// Loop recovers, then ticks until ctx ends. The first tick waits a full
// interval, so a restart isn't followed straight away by a backup.
func (s *Service) Loop(ctx context.Context) {
s.Recover(ctx)
t := time.NewTicker(tickEvery)
defer t.Stop()
for {
select {
case <-ctx.Done():
return
case <-t.C:
s.Tick(ctx)
}
}
}
// Close stops a running backup and waits for it to record how it ended.
func (s *Service) Close() {
s.mu.Lock()
s.closed = true
s.mu.Unlock()
s.cancel()
s.wg.Wait()
}
+281
View File
@@ -0,0 +1,281 @@
package backup_test
import (
"bytes"
"context"
"errors"
"fmt"
"io"
"log/slog"
"net/http"
"net/http/httptest"
"os"
"path/filepath"
"strings"
"sync"
"testing"
"time"
"github.com/jackc/pgx/v5/pgxpool"
"git.fabledsword.com/bvandeusen/minstrel/internal/backup"
)
func newService(t *testing.T) (*backup.Service, *pgxpool.Pool) {
t.Helper()
pool := newPool(t)
svc := backup.NewService(pool, slog.New(slog.NewTextHandler(io.Discard, nil)), "test-build")
t.Cleanup(svc.Close)
return svc, pool
}
// waitIdle waits for the running backup to finish, for at most 30 seconds.
func waitIdle(t *testing.T, svc *backup.Service) {
t.Helper()
deadline := time.Now().Add(30 * time.Second)
for !svc.Running().IsZero() {
if time.Now().After(deadline) {
t.Fatal("the backup is still running after 30s")
}
time.Sleep(20 * time.Millisecond)
}
}
func status(t *testing.T, svc *backup.Service) backup.Status {
t.Helper()
st, err := svc.Status(context.Background())
if err != nil {
t.Fatal(err)
}
return st
}
func TestABackupIsOffAndRefusesToStartUntilSetUp_Integration(t *testing.T) {
svc, _ := newService(t)
st := status(t, svc)
if st.State != backup.StateOff || st.Config.Enabled || st.NextRun != nil || len(st.Runs) != 0 {
t.Errorf("a fresh server's backup = %+v, want off with no runs", st)
}
if err := svc.Start(context.Background(), backup.TriggerManual); !errors.Is(err, backup.ErrNotReady) {
t.Errorf("Start = %v, want ErrNotReady", err)
}
if err := svc.Save(context.Background(), backup.Update{Enabled: true, Destination: backup.DestinationS3, RunHour: 3}); err != nil {
t.Fatal(err)
}
if st := status(t, svc); st.State != backup.StateIncomplete || len(st.Problems) != 4 {
t.Errorf("switched on with no bucket: %s %v", st.State, st.Problems)
}
}
func TestABackupRunsIntoAFolder_Integration(t *testing.T) {
svc, _ := newService(t)
dir := t.TempDir()
ctx := context.Background()
if err := svc.Save(ctx, backup.Update{Enabled: true, Destination: backup.DestinationFolder, Folder: dir, RunHour: 3}); err != nil {
t.Fatal(err)
}
if st := status(t, svc); st.State != backup.StateWaiting || st.NextRun == nil {
t.Errorf("set up, never run: %s next %v", st.State, st.NextRun)
}
if err := svc.Start(ctx, backup.TriggerManual); err != nil {
t.Fatal(err)
}
waitIdle(t, svc)
st := status(t, svc)
if st.State != backup.StateOK || len(st.Runs) != 1 || st.LastSuccess == nil {
t.Fatalf("after a run: %+v", st)
}
run := st.Runs[0]
if run.Status != "succeeded" || run.Trigger != backup.TriggerManual || run.FinishedAt == nil ||
filepath.Dir(run.Location) != filepath.Join(dir, "database") || run.Error != "" {
t.Fatalf("run = %+v", run)
}
f, err := os.Open(run.Location)
if err != nil {
t.Fatal(err)
}
defer func() { _ = f.Close() }()
info, _ := f.Stat()
if info.Size() != run.Bytes {
t.Errorf("recorded %d bytes, the file holds %d", run.Bytes, info.Size())
}
m, _, err := backup.Read(f)
if err != nil || m.Build != "test-build" {
t.Errorf("the backup reads back as %+v, %v", m, err)
}
res, err := svc.Test(ctx, backup.Update{Destination: backup.DestinationFolder, Folder: dir, RunHour: 3})
if err != nil || res.Backups != 1 || res.Latest == nil {
t.Errorf("Test = %+v, %v; want the one backup", res, err)
}
}
func TestAMissingFolderFailsTheRunAndSaysSo_Integration(t *testing.T) {
svc, _ := newService(t)
ctx := context.Background()
missing := filepath.Join(t.TempDir(), "not-mounted")
if err := svc.Save(ctx, backup.Update{Enabled: true, Destination: backup.DestinationFolder, Folder: missing, RunHour: 3}); err != nil {
t.Fatal(err)
}
if _, err := svc.Test(ctx, backup.Update{Destination: backup.DestinationFolder, Folder: missing}); err == nil {
t.Error("Test passed on a missing folder")
}
if err := svc.Start(ctx, backup.TriggerManual); err != nil {
t.Fatal(err)
}
waitIdle(t, svc)
st := status(t, svc)
if st.State != backup.StateFailing || st.FailingSince == nil || len(st.Runs) != 1 ||
!strings.Contains(st.Runs[0].Error, "isn't there") {
t.Errorf("after a failed run: %s %+v", st.State, st.Runs)
}
if _, err := os.Stat(missing); !os.IsNotExist(err) {
t.Error("the run created the missing folder")
}
}
// fakeBucket answers PUTs and lists like an S3 bucket. While hold is open a
// PUT waits on it.
type fakeBucket struct {
mu sync.Mutex
puts map[string][]byte
hold chan struct{}
}
func (b *fakeBucket) ServeHTTP(w http.ResponseWriter, r *http.Request) {
if !strings.HasPrefix(r.Header.Get("Authorization"), "AWS4-HMAC-SHA256 Credential=ak/") {
w.WriteHeader(http.StatusForbidden)
return
}
switch r.Method {
case http.MethodPut:
if b.hold != nil {
<-b.hold
}
body, _ := io.ReadAll(r.Body)
b.mu.Lock()
b.puts[r.URL.Path] = body
b.mu.Unlock()
case http.MethodGet:
b.mu.Lock()
defer b.mu.Unlock()
_, _ = fmt.Fprint(w, `<ListBucketResult><IsTruncated>false</IsTruncated>`)
for p, body := range b.puts {
_, _ = fmt.Fprintf(w, `<Contents><Key>%s</Key><Size>%d</Size><LastModified>2026-11-02T03:00:05.000Z</LastModified></Contents>`,
strings.TrimPrefix(p, "/minstrel/"), len(body))
}
_, _ = fmt.Fprint(w, `</ListBucketResult>`)
}
}
func bucketUpdate(endpoint string) backup.Update {
return backup.Update{Enabled: true, Destination: backup.DestinationS3, Endpoint: endpoint,
Bucket: "minstrel", AccessKey: "ak", SecretKey: "sk", RunHour: 3}
}
func TestABackupRunsIntoABucket_Integration(t *testing.T) {
svc, pool := newService(t)
fake := &fakeBucket{puts: map[string][]byte{}}
srv := httptest.NewServer(fake)
defer srv.Close()
ctx := context.Background()
if err := svc.Save(ctx, bucketUpdate(srv.URL)); err != nil {
t.Fatal(err)
}
var stored string
if err := pool.QueryRow(ctx, "SELECT secret_key FROM backup_settings").Scan(&stored); err != nil {
t.Fatal(err)
}
if !strings.HasPrefix(stored, "v1.") {
t.Fatalf("the secret key is stored as %q, want it sealed", stored)
}
if cfg, _ := svc.Config(ctx); !cfg.HasSecretKey {
t.Error("the config doesn't say a secret key is stored")
}
if err := svc.Start(ctx, backup.TriggerManual); err != nil {
t.Fatal(err)
}
waitIdle(t, svc)
st := status(t, svc)
if st.State != backup.StateOK || len(st.Runs) != 1 || st.Runs[0].Status != "succeeded" {
t.Fatalf("after a run: %s %+v", st.State, st.Runs)
}
key := st.Runs[0].Location
if !strings.HasPrefix(key, "database/minstrel-") || !backup.IsFileName(filepath.Base(key)) {
t.Errorf("stored under %q", key)
}
fake.mu.Lock()
body := fake.puts["/minstrel/"+key]
fake.mu.Unlock()
if int64(len(body)) != st.Runs[0].Bytes {
t.Errorf("the bucket got %d bytes, the run recorded %d", len(body), st.Runs[0].Bytes)
}
if _, _, err := backup.Read(bytes.NewReader(body)); err != nil {
t.Errorf("the uploaded backup doesn't read: %v", err)
}
// The test sends no secret key: it uses the stored one, and only lists.
u := bucketUpdate(srv.URL)
u.SecretKey = ""
res, err := svc.Test(ctx, u)
if err != nil || res.Backups != 1 {
t.Errorf("Test = %+v, %v; want the one backup", res, err)
}
fake.mu.Lock()
defer fake.mu.Unlock()
if len(fake.puts) != 1 {
t.Errorf("the test wrote to the bucket: %d objects", len(fake.puts))
}
}
func TestOneBackupAtATime_Integration(t *testing.T) {
svc, _ := newService(t)
fake := &fakeBucket{puts: map[string][]byte{}, hold: make(chan struct{})}
srv := httptest.NewServer(fake)
defer srv.Close()
ctx := context.Background()
if err := svc.Save(ctx, bucketUpdate(srv.URL)); err != nil {
t.Fatal(err)
}
if err := svc.Start(ctx, backup.TriggerManual); err != nil {
t.Fatal(err)
}
if err := svc.Start(ctx, backup.TriggerManual); !errors.Is(err, backup.ErrRunning) {
t.Errorf("a second Start = %v, want ErrRunning", err)
}
if st := status(t, svc); st.RunningSince == nil {
t.Error("the status doesn't say a backup is running")
}
close(fake.hold)
waitIdle(t, svc)
}
func TestRecoverFailsARunTheLastShutdownCutOff_Integration(t *testing.T) {
svc, pool := newService(t)
ctx := context.Background()
if _, err := pool.Exec(ctx, "INSERT INTO backup_runs (trigger) VALUES ('schedule')"); err != nil {
t.Fatal(err)
}
svc.Recover(ctx)
st := status(t, svc)
if len(st.Runs) != 1 || st.Runs[0].Status != "failed" || st.Runs[0].FinishedAt == nil ||
!strings.Contains(st.Runs[0].Error, "server stopped") {
t.Errorf("runs = %+v", st.Runs)
}
}
func TestSaveRefusesABadValueAndKeepsTheOldSettings_Integration(t *testing.T) {
svc, _ := newService(t)
ctx := context.Background()
err := svc.Save(ctx, backup.Update{Enabled: true, Destination: backup.DestinationFolder, Folder: "relative/path"})
var fe *backup.FieldError
if !errors.As(err, &fe) || fe.Field != "folder" {
t.Fatalf("err = %v, want a FieldError on folder", err)
}
if cfg, _ := svc.Config(ctx); cfg.Enabled || cfg.Folder != "" {
t.Errorf("a refused save changed the settings: %+v", cfg)
}
}
+171
View File
@@ -0,0 +1,171 @@
package backup
import (
"bytes"
"context"
"errors"
"io"
"strings"
"testing"
"time"
"git.fabledsword.com/bvandeusen/minstrel/internal/db/dbq"
)
func at(s string) time.Time {
t, err := time.Parse(time.RFC3339, s)
if err != nil {
panic(err)
}
return t
}
func TestDueRunsAtTheHourCatchesUpAndBacksOffAfterAFailure(t *testing.T) {
cases := []struct {
name string
now string
t runTimes
want string
}{
{"never run: the slot that passed is due", "2026-11-02T10:00:00Z",
runTimes{}, "2026-11-02T03:00:00Z"},
{"ran at the hour: tomorrow's slot", "2026-11-02T10:00:00Z",
runTimes{lastSuccess: at("2026-11-02T03:00:05Z"), lastAttempt: at("2026-11-02T03:00:05Z")}, "2026-11-03T03:00:00Z"},
{"before today's hour, yesterday's ran: today's slot", "2026-11-02T02:00:00Z",
runTimes{lastSuccess: at("2026-11-01T03:00:05Z"), lastAttempt: at("2026-11-01T03:00:05Z")}, "2026-11-02T03:00:00Z"},
{"server was down at the hour: catches up", "2026-11-02T10:00:00Z",
runTimes{lastSuccess: at("2026-11-01T03:00:05Z"), lastAttempt: at("2026-11-01T03:00:05Z")}, "2026-11-02T03:00:00Z"},
{"first failure: an hour later", "2026-11-02T03:30:00Z",
runTimes{lastAttempt: at("2026-11-02T03:00:00Z"), failingSince: at("2026-11-02T03:00:00Z")}, "2026-11-02T04:00:00Z"},
{"failing for two hours: two hours later", "2026-11-02T05:30:00Z",
runTimes{lastAttempt: at("2026-11-02T05:00:00Z"), failingSince: at("2026-11-02T03:00:00Z")}, "2026-11-02T07:00:00Z"},
{"a retry past the next slot waits for the slot", "2026-11-02T19:30:00Z",
runTimes{lastAttempt: at("2026-11-02T19:00:00Z"), failingSince: at("2026-11-02T03:00:00Z")}, "2026-11-03T03:00:00Z"},
}
for _, c := range cases {
if got := due(at(c.now), 3, c.t); !got.Equal(at(c.want)) {
t.Errorf("%s: due = %s, want %s", c.name, got.Format(time.RFC3339), c.want)
}
}
}
func TestTheHourIsUTC(t *testing.T) {
// 22:00 at UTC-5 is 03:00 UTC the next day, which is that day's slot.
now := time.Date(2026, 11, 1, 22, 0, 0, 0, time.FixedZone("x", -5*3600))
if got := slotBefore(now, 3); !got.Equal(at("2026-11-02T03:00:00Z")) {
t.Errorf("slot = %s", got)
}
}
func TestVerdict(t *testing.T) {
now := at("2026-11-02T10:00:00Z")
on := dbq.BackupSetting{Enabled: true}
good := runTimes{lastSuccess: at("2026-11-02T03:00:00Z"), lastAttempt: at("2026-11-02T03:00:00Z")}
cases := []struct {
name string
st dbq.BackupSetting
probs []string
t runTimes
want string
}{
{"switched off, even with problems and failures", dbq.BackupSetting{}, []string{"x"}, runTimes{failingSince: now}, StateOff},
{"on but not set up", on, []string{"x"}, good, StateIncomplete},
{"the latest run failed", on, nil, runTimes{lastSuccess: good.lastSuccess, failingSince: now}, StateFailing},
{"nothing has run yet", on, nil, runTimes{}, StateWaiting},
{"the latest backup is old", on, nil, runTimes{lastSuccess: now.Add(-27 * time.Hour)}, StateOverdue},
{"a day and a bit is still fine", on, nil, runTimes{lastSuccess: now.Add(-25 * time.Hour)}, StateOK},
{"fine", on, nil, good, StateOK},
}
for _, c := range cases {
if got := verdict(c.st, c.probs, c.t, now); got != c.want {
t.Errorf("%s: %s, want %s", c.name, got, c.want)
}
}
}
func TestProblemsNameWhatIsMissing(t *testing.T) {
if p := problems(dbq.BackupSetting{Destination: DestinationS3}); len(p) != 4 {
t.Errorf("an empty bucket: %v, want endpoint, name, access key and secret key", p)
}
if p := problems(dbq.BackupSetting{Destination: DestinationS3, Endpoint: "https://s3.example.com", Bucket: "b", AccessKey: "a", SecretKey: "v1.x"}); len(p) != 0 {
t.Errorf("a whole bucket: %v", p)
}
if p := problems(dbq.BackupSetting{Destination: DestinationFolder}); len(p) != 1 {
t.Errorf("an empty folder: %v", p)
}
if p := problems(dbq.BackupSetting{Destination: DestinationFolder, Folder: "/backups"}); len(p) != 0 {
t.Errorf("a folder: %v", p)
}
}
func TestMergeRefusesWhatCannotBeKept(t *testing.T) {
ok := Update{Destination: DestinationS3, RunHour: 3}
for field, u := range map[string]func(Update) Update{
"destination": func(u Update) Update { u.Destination = "ftp"; return u },
"run_hour": func(u Update) Update { u.RunHour = 24; return u },
"upload_limit_kib": func(u Update) Update { u.UploadLimitKiB = -1; return u },
"folder": func(u Update) Update { u.Folder = "backups"; return u },
"endpoint": func(u Update) Update { u.Endpoint = "s3.example.com"; return u },
} {
_, err := merge(dbq.BackupSetting{}, u(ok))
var fe *FieldError
if !errors.As(err, &fe) || fe.Field != field {
t.Errorf("%s: err = %v, want a FieldError on it", field, err)
}
}
}
func TestMergeTidiesAndKeepsTheStoredSecret(t *testing.T) {
st, err := merge(dbq.BackupSetting{SecretKey: "v1.stored"}, Update{
Enabled: true, Destination: DestinationS3, Endpoint: " https://s3.example.com/ ",
Bucket: " minstrel ", AccessKey: " ak ", RunHour: 4, UploadLimitKiB: 512, Folder: "/srv/backups/",
})
if err != nil {
t.Fatal(err)
}
if st.Endpoint != "https://s3.example.com" || st.Bucket != "minstrel" || st.AccessKey != "ak" ||
st.SecretKey != "v1.stored" || st.RunHour != 4 || st.UploadLimitKib != 512 || st.Folder != "/srv/backups" || !st.Enabled {
t.Errorf("merged = %+v", st)
}
}
func TestThrottleKeepsToItsRate(t *testing.T) {
data := bytes.Repeat([]byte("x"), 20*1024)
start := time.Now()
got, err := io.ReadAll(Throttle(context.Background(), bytes.NewReader(data), 100*1024))
if err != nil || !bytes.Equal(got, data) {
t.Fatalf("read %d bytes, err %v", len(got), err)
}
// 20 KiB at 100 KiB a second takes at least a fifth of a second.
if took := time.Since(start); took < 180*time.Millisecond {
t.Errorf("took %s, want at least 200ms", took)
}
}
func TestThrottleWithNoLimitIsTheReaderItself(t *testing.T) {
r := strings.NewReader("x")
if Throttle(context.Background(), r, 0) != io.Reader(r) {
t.Error("a zero limit wrapped the reader")
}
}
func TestThrottleStopsWhenItsContextEnds(t *testing.T) {
ctx, cancel := context.WithCancel(context.Background())
cancel()
_, err := io.ReadAll(Throttle(ctx, bytes.NewReader(make([]byte, 4096)), 1024))
if !errors.Is(err, context.Canceled) {
t.Errorf("err = %v, want context.Canceled", err)
}
}
func TestDescribeSaysWhyARunStopped(t *testing.T) {
if got := describe(context.DeadlineExceeded); !strings.Contains(got, "longer than 30 minutes") {
t.Errorf("deadline: %q", got)
}
if got := describe(context.Canceled); !strings.Contains(got, "server stopped") {
t.Errorf("canceled: %q", got)
}
if got := describe(errors.New("backup: the folder /x isn't there")); got != "the folder /x isn't there" {
t.Errorf("other: %q", got)
}
}
+119
View File
@@ -0,0 +1,119 @@
package backup
import (
"crypto/hmac"
"crypto/sha256"
"encoding/hex"
"net/http"
"net/url"
"slices"
"strings"
"time"
)
// emptySHA256 is the payload hash of a request with no body.
const emptySHA256 = "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855"
// sign adds AWS Signature Version 4 headers for S3 to req: X-Amz-Date,
// X-Amz-Content-Sha256 and Authorization. Every header already on the
// request is signed, along with the host.
func sign(req *http.Request, payloadSHA256, accessKey, secretKey, region string, at time.Time) {
at = at.UTC()
stamp := at.Format("20060102T150405Z")
date := at.Format("20060102")
req.Header.Set("X-Amz-Date", stamp)
req.Header.Set("X-Amz-Content-Sha256", payloadSHA256)
host := req.Host
if host == "" {
host = req.URL.Host
}
headers := map[string]string{"host": host}
for name, values := range req.Header {
headers[strings.ToLower(name)] = strings.TrimSpace(strings.Join(values, ","))
}
names := make([]string, 0, len(headers))
for name := range headers {
names = append(names, name)
}
slices.Sort(names)
var canonHeaders strings.Builder
for _, name := range names {
canonHeaders.WriteString(name + ":" + headers[name] + "\n")
}
signed := strings.Join(names, ";")
canonical := strings.Join([]string{
req.Method,
canonicalPath(req.URL),
canonicalQuery(req.URL.Query()),
canonHeaders.String(),
signed,
payloadSHA256,
}, "\n")
scope := date + "/" + region + "/s3/aws4_request"
toSign := "AWS4-HMAC-SHA256\n" + stamp + "\n" + scope + "\n" + hexSHA256([]byte(canonical))
key := hmacSHA256([]byte("AWS4"+secretKey), date)
key = hmacSHA256(key, region)
key = hmacSHA256(key, "s3")
key = hmacSHA256(key, "aws4_request")
signature := hex.EncodeToString(hmacSHA256(key, toSign))
req.Header.Set("Authorization", "AWS4-HMAC-SHA256 Credential="+accessKey+"/"+scope+
", SignedHeaders="+signed+", Signature="+signature)
}
// canonicalPath is the request path with each segment URI-encoded once.
func canonicalPath(u *url.URL) string {
p := u.EscapedPath()
if p == "" {
return "/"
}
if unescaped, err := url.PathUnescape(p); err == nil {
p = unescaped
}
segments := strings.Split(p, "/")
for i, s := range segments {
segments[i] = uriEncode(s)
}
return strings.Join(segments, "/")
}
// canonicalQuery is the query sorted by name, names and values URI-encoded.
func canonicalQuery(q url.Values) string {
pairs := make([]string, 0, len(q))
for name, values := range q {
for _, v := range values {
pairs = append(pairs, uriEncode(name)+"="+uriEncode(v))
}
}
slices.Sort(pairs)
return strings.Join(pairs, "&")
}
// uriEncode percent-encodes everything but the unreserved characters, as
// SigV4 requires; url.QueryEscape differs on space and '~'.
func uriEncode(s string) string {
var b strings.Builder
for i := 0; i < len(s); i++ {
c := s[i]
if c >= 'A' && c <= 'Z' || c >= 'a' && c <= 'z' || c >= '0' && c <= '9' || c == '-' || c == '_' || c == '.' || c == '~' {
b.WriteByte(c)
continue
}
b.WriteString("%" + strings.ToUpper(hex.EncodeToString([]byte{c})))
}
return b.String()
}
func hmacSHA256(key []byte, msg string) []byte {
m := hmac.New(sha256.New, key)
m.Write([]byte(msg))
return m.Sum(nil)
}
func hexSHA256(b []byte) string {
sum := sha256.Sum256(b)
return hex.EncodeToString(sum[:])
}
+114
View File
@@ -0,0 +1,114 @@
package backup
import (
"bytes"
"context"
"encoding/json"
"fmt"
"strings"
"github.com/jackc/pgx/v5"
"github.com/jackc/pgx/v5/pgxpool"
)
// Contents is a backup's data: the schema version that wrote it, and each
// table's rows as JSON objects with their keys sorted, in primary-key order.
// An unchanged row is the same bytes in every backup.
type Contents struct {
Schema int64
Tables map[string][][]byte
}
// Snapshot reads every table in Tables as of one moment.
func Snapshot(ctx context.Context, pool *pgxpool.Pool) (Contents, error) {
c := Contents{Tables: make(map[string][][]byte, len(Tables))}
tx, err := pool.BeginTx(ctx, pgx.TxOptions{IsoLevel: pgx.RepeatableRead, AccessMode: pgx.ReadOnly})
if err != nil {
return c, fmt.Errorf("backup: begin: %w", err)
}
defer func() { _ = tx.Rollback(ctx) }()
if c.Schema, err = schemaVersion(ctx, tx); err != nil {
return c, err
}
for _, t := range Tables {
lines, err := readTable(ctx, tx, t)
if err != nil {
return c, err
}
c.Tables[t.Name] = lines
}
return c, nil
}
func readTable(ctx context.Context, tx pgx.Tx, t Table) ([][]byte, error) {
pk, err := primaryKey(ctx, tx, t.Name)
if err != nil {
return nil, err
}
order := make([]string, len(pk))
for i, col := range pk {
order[i] = "t." + pgx.Identifier{col}.Sanitize()
}
omit := t.Omit
if omit == nil {
omit = []string{}
}
rows, err := tx.Query(ctx, fmt.Sprintf("SELECT to_jsonb(t) - $1::text[] FROM %s t ORDER BY %s",
pgx.Identifier{t.Name}.Sanitize(), strings.Join(order, ", ")), omit)
if err != nil {
return nil, fmt.Errorf("backup: read %s: %w", t.Name, err)
}
lines, err := pgx.CollectRows(rows, func(row pgx.CollectableRow) ([]byte, error) {
var raw []byte
if err := row.Scan(&raw); err != nil {
return nil, err
}
return canonical(raw)
})
if err != nil {
return nil, fmt.Errorf("backup: read %s: %w", t.Name, err)
}
return lines, nil
}
// canonical re-encodes a row with its keys sorted. Numbers pass through as
// written, so a bigint or a float stays exact.
func canonical(raw []byte) ([]byte, error) {
dec := json.NewDecoder(bytes.NewReader(raw))
dec.UseNumber()
var row map[string]any
if err := dec.Decode(&row); err != nil {
return nil, err
}
return json.Marshal(row)
}
// primaryKey returns a table's primary-key columns in key order.
func primaryKey(ctx context.Context, tx pgx.Tx, table string) ([]string, error) {
rows, err := tx.Query(ctx, `
SELECT a.attname
FROM pg_index i
JOIN pg_attribute a ON a.attrelid = i.indrelid AND a.attnum = ANY (i.indkey)
WHERE i.indrelid = to_regclass($1) AND i.indisprimary
ORDER BY array_position(i.indkey, a.attnum)`, table)
if err != nil {
return nil, fmt.Errorf("backup: read %s's primary key: %w", table, err)
}
cols, err := pgx.CollectRows(rows, pgx.RowTo[string])
if err != nil {
return nil, fmt.Errorf("backup: read %s's primary key: %w", table, err)
}
if len(cols) == 0 {
return nil, fmt.Errorf("backup: %s has no primary key to order its rows by", table)
}
return cols, nil
}
func schemaVersion(ctx context.Context, tx pgx.Tx) (int64, error) {
var v int64
if err := tx.QueryRow(ctx, "SELECT version FROM schema_migrations").Scan(&v); err != nil {
return 0, fmt.Errorf("backup: read the schema version: %w", err)
}
return v, nil
}
+198
View File
@@ -0,0 +1,198 @@
// Package backup reads Minstrel's irreplaceable data out of Postgres as JSON
// Lines, one table at a time, and loads it back into an empty instance.
//
// What goes in is everything a person made or decided: accounts, likes, play
// and skip history, playlists, follows, wants and requests, download history,
// admin decisions, and settings. What stays out is everything the restored
// server rebuilds by itself: what it computes from the music files, what it
// fetches again from MusicBrainz, ListenBrainz and the other providers, and
// logs of past runs. Excluded says why for each, and the coverage test fails
// on a table that is in neither list.
package backup
// Table is one table a backup holds.
//
// Omit names columns left out of the file. They restore to their defaults:
// - a credential Minstrel stores sealed, which only the server holding the
// stream secret could open, so the restored server asks for it again;
// - a marker saying work on an excluded table is done (a tag source, an art
// source), so the restored server does that work again rather than
// believing it has the result.
type Table struct {
Name string
Omit []string
}
// Tables is every table a backup holds, in the order a restore loads them:
// each after every table it references. There are no self-referencing
// tables, so no table needs its rows loaded in depth order.
var Tables = []Table{
{Name: "users", Omit: []string{"subsonic_password", "listenbrainz_token"}},
// The library's identity. Every like, play and playlist entry hangs off a
// track id, so the rows travel even though a scan could rebuild most of
// their columns; the scan after a restore finds each file at its path and
// keeps its id.
{Name: "artists", Omit: []string{"artist_thumb_path", "artist_fanart_path", "artist_art_source", "artist_art_sources_version"}},
{Name: "albums", Omit: []string{"cover_art_path", "cover_art_source", "cover_art_sources_version"}},
{Name: "tracks", Omit: []string{"tag_source", "tag_sources_version"}},
// Settings.
{Name: "library_settings"},
{Name: "network_settings"},
{Name: "registration_settings"},
{Name: "smtp_config", Omit: []string{"password"}},
{Name: "notification_email_settings"},
{Name: "acoustid_settings", Omit: []string{"api_key"}},
{Name: "fingerprint_settings"},
{Name: "loudness_settings"},
{Name: "cover_art_provider_settings", Omit: []string{"api_key"}},
{Name: "cover_art_sources_meta"},
{Name: "tag_provider_settings", Omit: []string{"api_key"}},
{Name: "tag_sources_meta"},
{Name: "acquisition_services", Omit: []string{"api_key", "password"}},
{Name: "acquisition_settings"},
{Name: "reacquisition_settings"},
{Name: "discover_tuning"},
{Name: "taste_tuning"},
{Name: "recommendation_weight_profiles"},
{Name: "recommendation_tuning_audit"},
// Follows, wants, requests and downloads.
{Name: "release_type_profiles"},
{Name: "follows"},
{Name: "follow_seen_groups"},
{Name: "wants"},
{Name: "grabs"},
{Name: "download_interventions"},
{Name: "release_blocks"},
{Name: "requests"},
// Listening history and likes.
{Name: "play_sessions"},
{Name: "play_events"},
{Name: "skip_events"},
{Name: "scrobble_queue"},
{Name: "contextual_likes"},
{Name: "general_likes"},
{Name: "general_likes_albums"},
{Name: "general_likes_artists"},
// Playlists and what is built from history. The taste profile and the
// "you might like" rows are rebuilt nightly from the history above; they
// travel so a restored Home is full before that first nightly pass.
{Name: "playlists"},
{Name: "playlist_tracks"},
{Name: "system_playlist_runs"},
{Name: "system_playlist_rotation_state"},
{Name: "taste_profile_artists"},
{Name: "taste_profile_tags"},
{Name: "taste_profile_eras"},
{Name: "taste_profile_moods"},
{Name: "you_might_like_albums"},
{Name: "you_might_like_artists"},
// Per-user preferences and notifications.
{Name: "user_normalization_prefs"},
{Name: "user_notification_prefs"},
{Name: "user_notification_email_state"},
{Name: "user_notifications"},
{Name: "suggestion_snoozes"},
{Name: "quarantine"},
{Name: "quarantine_actions"},
// Admin decisions: an album taken off the wanted listing stays off, and a
// dismissed duplicate group stays dismissed when the sweep finds it again.
{Name: "library_album_matches"},
{Name: "duplicate_sweeps"},
{Name: "duplicate_groups"},
{Name: "duplicate_group_members"},
// The library change feed is how the apps sync, including a first sync,
// so a restored server carries it on from where the backup left it.
{Name: "library_changes"},
{Name: "library_changes_trimmed"},
{Name: "audit_log"},
{Name: "playback_errors"},
}
// Excluded is every table a backup leaves out, with the reason.
var Excluded = map[string]string{
"schema_migrations": "the restored server's own migrations write it; the backup's schema version is in its manifest",
"sessions": "sign-ins; everyone signs in again on the restored server",
"password_resets": "one-time reset links, which expire within hours",
"user_invites": "open sign-up links; an admin sends an invite again from the restored server",
"track_fingerprints": "computed from the music files",
"track_loudness": "computed from the music files",
"album_loudness": "computed from the music files",
"track_acoustid_lookups": "looked up again from each track's fingerprint",
"track_tags": "fetched again from the tag providers",
"candidate_artist_tags": "fetched again from the tag providers",
"candidate_artist_tag_state": "fetched again from the tag providers",
"mb_artists": "MusicBrainz cache, fetched again",
"mb_release_groups": "MusicBrainz cache, fetched again",
"mb_releases": "MusicBrainz cache, fetched again",
"mb_release_tracks": "MusicBrainz cache, fetched again",
"mb_artist_release_groups": "MusicBrainz cache, fetched again",
"listenbrainz_similar_recordings": "similarity, fetched again from ListenBrainz",
"track_similarity": "similarity, fetched again from the similarity providers",
"track_similarity_fetches": "similarity, fetched again from the similarity providers",
"artist_similarity": "similarity, fetched again from the similarity providers",
"artist_similarity_fetches": "similarity, fetched again from the similarity providers",
"artist_similarity_unmatched": "similarity, fetched again from the similarity providers",
"want_candidates": "the last search's results; the next search finds them again",
"scan_runs": "a log of past library scans",
"diagnostic_events": "debug-mode diagnostics",
"backup_settings": "holds the bucket's secret key; point the restored server at its destination again",
"backup_runs": "this server's own backup history",
"backup_music_files": "which music files this server has sent; the files themselves are beside the database in the destination",
}
// carried reports whether a table is in Tables.
func carried(name string) bool {
for _, t := range Tables {
if t.Name == name {
return true
}
}
return false
}
// Coverage checks a schema's tables against Tables and Excluded. It returns
// one line per problem: a table in neither list, in both, or listed twice,
// and a listed table the schema doesn't have.
func Coverage(schema []string) []string {
var problems []string
inSchema := map[string]bool{}
for _, name := range schema {
inSchema[name] = true
_, excluded := Excluded[name]
switch {
case carried(name) && excluded:
problems = append(problems, name+": both backed up and excluded")
case !carried(name) && !excluded:
problems = append(problems, name+": neither backed up nor excluded; add it to Tables, or to Excluded with the reason")
}
}
seen := map[string]bool{}
for _, t := range Tables {
if seen[t.Name] {
problems = append(problems, t.Name+": listed twice in Tables")
}
seen[t.Name] = true
if !inSchema[t.Name] {
problems = append(problems, t.Name+": in Tables but not in the schema")
}
}
for name := range Excluded {
if !inSchema[name] {
problems = append(problems, name+": in Excluded but not in the schema")
}
}
return problems
}
+50
View File
@@ -0,0 +1,50 @@
package backup
import (
"slices"
"strings"
"testing"
)
func everyListedTable() []string {
var names []string
for _, t := range Tables {
names = append(names, t.Name)
}
for name := range Excluded {
names = append(names, name)
}
return names
}
func TestCoverageAcceptsASchemaThatMatchesTheLists(t *testing.T) {
if problems := Coverage(everyListedTable()); len(problems) != 0 {
t.Fatalf("problems = %v, want none", problems)
}
}
func TestCoverageNamesATableInNeitherList(t *testing.T) {
problems := Coverage(append(everyListedTable(), "new_table"))
if len(problems) != 1 || !strings.HasPrefix(problems[0], "new_table: neither") {
t.Fatalf("problems = %v, want one naming new_table", problems)
}
}
func TestCoverageNamesAListedTableTheSchemaLacks(t *testing.T) {
schema := slices.DeleteFunc(everyListedTable(), func(n string) bool { return n == "playlists" || n == "scan_runs" })
problems := Coverage(schema)
if len(problems) != 2 {
t.Fatalf("problems = %v, want playlists and scan_runs named", problems)
}
}
func TestEveryExclusionHasAReason(t *testing.T) {
for name, why := range Excluded {
if strings.TrimSpace(why) == "" {
t.Errorf("%s is excluded without a reason", name)
}
if carried(name) {
t.Errorf("%s is both backed up and excluded", name)
}
}
}
+50
View File
@@ -0,0 +1,50 @@
package backup
import (
"context"
"io"
"time"
)
// Throttle returns a reader that reads r no faster than perSecond bytes a
// second, so an upload leaves room on the connection for everything else.
// perSecond of 0 or less returns r unchanged. A wait ends early when ctx does,
// and the read then returns ctx's error.
func Throttle(ctx context.Context, r io.Reader, perSecond int64) io.Reader {
if perSecond <= 0 {
return r
}
return &throttled{ctx: ctx, r: r, perSecond: perSecond}
}
type throttled struct {
ctx context.Context
r io.Reader
perSecond int64
start time.Time
sent int64
}
func (t *throttled) Read(p []byte) (int, error) {
if t.start.IsZero() {
t.start = time.Now()
}
// Reads are cut to a tenth of a second's worth, so the pace is even
// rather than a burst followed by a long pause.
if chunk := max(t.perSecond/10, 1); int64(len(p)) > chunk {
p = p[:chunk]
}
n, err := t.r.Read(p)
t.sent += int64(n)
at := t.start.Add(time.Duration(float64(t.sent) / float64(t.perSecond) * float64(time.Second)))
if wait := time.Until(at); wait > 0 {
timer := time.NewTimer(wait)
defer timer.Stop()
select {
case <-t.ctx.Done():
return n, t.ctx.Err()
case <-timer.C:
}
}
return n, err
}
+8 -3
View File
@@ -13,6 +13,7 @@ import (
"github.com/jackc/pgx/v5/pgxpool"
"git.fabledsword.com/bvandeusen/minstrel/internal/db/dbq"
"git.fabledsword.com/bvandeusen/minstrel/internal/secret"
)
// SettingsService reconciles the registry of compiled-in providers
@@ -112,7 +113,7 @@ func (s *SettingsService) reconcile(ctx context.Context) error {
}
key := ""
if row.ApiKey != nil {
key = *row.ApiKey
key = secret.Reveal(secret.CoverArtAPIKey.Purpose(), row.ProviderID, *row.ApiKey)
}
s.enabledIDs[p.ID()] = row.Enabled
s.apiKeys[p.ID()] = key
@@ -204,11 +205,15 @@ func (s *SettingsService) UpdateProvider(ctx context.Context, providerID string,
// (apiKeyChanged flag) and $4 (api_key value). The actual generated
// field for $4 is ApiKey *string.
apiKeyChanged := patch.APIKey != nil
sealedKey, err := secret.SealPtr(secret.CoverArtAPIKey.Purpose(), providerID, patch.APIKey)
if err != nil {
return false, fmt.Errorf("seal provider api key: %w", err)
}
if err := q.UpdateProviderSettings(ctx, dbq.UpdateProviderSettingsParams{
ProviderID: providerID,
Enabled: enabledVal,
Column3: apiKeyChanged,
ApiKey: patch.APIKey,
ApiKey: sealedKey,
}); err != nil {
return false, fmt.Errorf("update provider settings: %w", err)
}
@@ -225,7 +230,7 @@ func (s *SettingsService) UpdateProvider(ctx context.Context, providerID string,
for _, r := range rows {
key := ""
if r.ApiKey != nil {
key = *r.ApiKey
key = secret.Reveal(secret.CoverArtAPIKey.Purpose(), r.ProviderID, *r.ApiKey)
}
s.enabledIDs[r.ProviderID] = r.Enabled
s.apiKeys[r.ProviderID] = key
+389
View File
@@ -0,0 +1,389 @@
// Code generated by sqlc. DO NOT EDIT.
// versions:
// sqlc v1.31.1
// source: backups.sql
package dbq
import (
"context"
"github.com/jackc/pgx/v5/pgtype"
)
const backupMusicProgress = `-- name: BackupMusicProgress :one
SELECT count(*)::bigint AS total_files,
COALESCE(sum(t.file_size), 0)::bigint AS total_bytes,
count(*) FILTER (WHERE m.error = '' AND m.size = t.file_size AND t.updated_at <= m.checked_at)::bigint AS sent_files,
COALESCE(sum(t.file_size) FILTER (WHERE m.error = '' AND m.size = t.file_size AND t.updated_at <= m.checked_at), 0)::bigint AS sent_bytes,
count(*) FILTER (WHERE m.error <> '' AND m.size = t.file_size AND t.updated_at <= m.checked_at)::bigint AS refused_files
FROM tracks t
LEFT JOIN backup_music_files m ON m.track_id = t.id
WHERE t.missing_since IS NULL
`
type BackupMusicProgressRow struct {
TotalFiles int64
TotalBytes int64
SentFiles int64
SentBytes int64
RefusedFiles int64
}
// How much of the library the backup holds as it is now.
func (q *Queries) BackupMusicProgress(ctx context.Context) (BackupMusicProgressRow, error) {
row := q.db.QueryRow(ctx, backupMusicProgress)
var i BackupMusicProgressRow
err := row.Scan(
&i.TotalFiles,
&i.TotalBytes,
&i.SentFiles,
&i.SentBytes,
&i.RefusedFiles,
)
return i, err
}
const backupRunTimes = `-- name: BackupRunTimes :one
SELECT
COALESCE((SELECT max(started_at) FROM backup_runs WHERE status = 'succeeded'),
'epoch'::timestamptz)::timestamptz AS last_success,
COALESCE((SELECT max(started_at) FROM backup_runs),
'epoch'::timestamptz)::timestamptz AS last_attempt,
COALESCE((SELECT min(started_at) FROM backup_runs
WHERE status = 'failed'
AND started_at > COALESCE((SELECT max(started_at) FROM backup_runs WHERE status = 'succeeded'),
'epoch'::timestamptz)),
'epoch'::timestamptz)::timestamptz AS failing_since
`
type BackupRunTimesRow struct {
LastSuccess pgtype.Timestamptz
LastAttempt pgtype.Timestamptz
FailingSince pgtype.Timestamptz
}
// The times the verdict turns on. A run that never happened is the epoch,
// so the row always exists. failing_since is the first failure after the last
// success, and the epoch when the latest finished run succeeded.
func (q *Queries) BackupRunTimes(ctx context.Context) (BackupRunTimesRow, error) {
row := q.db.QueryRow(ctx, backupRunTimes)
var i BackupRunTimesRow
err := row.Scan(&i.LastSuccess, &i.LastAttempt, &i.FailingSince)
return i, err
}
const failUnfinishedBackupRuns = `-- name: FailUnfinishedBackupRuns :execrows
UPDATE backup_runs
SET status = 'failed', finished_at = now(),
error = 'The server stopped while the backup was running.'
WHERE status = 'running'
`
func (q *Queries) FailUnfinishedBackupRuns(ctx context.Context) (int64, error) {
result, err := q.db.Exec(ctx, failUnfinishedBackupRuns)
if err != nil {
return 0, err
}
return result.RowsAffected(), nil
}
const finishBackupRun = `-- name: FinishBackupRun :exec
UPDATE backup_runs
SET status = $2,
finished_at = now(),
location = $3,
bytes = $4,
error = $5,
music_files = $6,
music_bytes = $7,
music_note = $8
WHERE id = $1
`
type FinishBackupRunParams struct {
ID pgtype.UUID
Status string
Location string
Bytes int64
Error string
MusicFiles int32
MusicBytes int64
MusicNote string
}
func (q *Queries) FinishBackupRun(ctx context.Context, arg FinishBackupRunParams) error {
_, err := q.db.Exec(ctx, finishBackupRun,
arg.ID,
arg.Status,
arg.Location,
arg.Bytes,
arg.Error,
arg.MusicFiles,
arg.MusicBytes,
arg.MusicNote,
)
return err
}
const getBackupSettings = `-- name: GetBackupSettings :one
SELECT id, enabled, destination, folder, endpoint, bucket, access_key, secret_key, run_hour, upload_limit_kib, updated_at, include_music FROM backup_settings WHERE id = true
`
// Minstrel's own backup: its settings and its runs.
func (q *Queries) GetBackupSettings(ctx context.Context) (BackupSetting, error) {
row := q.db.QueryRow(ctx, getBackupSettings)
var i BackupSetting
err := row.Scan(
&i.ID,
&i.Enabled,
&i.Destination,
&i.Folder,
&i.Endpoint,
&i.Bucket,
&i.AccessKey,
&i.SecretKey,
&i.RunHour,
&i.UploadLimitKib,
&i.UpdatedAt,
&i.IncludeMusic,
)
return i, err
}
const listBackupRuns = `-- name: ListBackupRuns :many
SELECT id, trigger, status, started_at, finished_at, location, bytes, error,
music_files, music_bytes, music_note
FROM backup_runs
ORDER BY started_at DESC
LIMIT $1
`
func (q *Queries) ListBackupRuns(ctx context.Context, limit int32) ([]BackupRun, error) {
rows, err := q.db.Query(ctx, listBackupRuns, limit)
if err != nil {
return nil, err
}
defer rows.Close()
var items []BackupRun
for rows.Next() {
var i BackupRun
if err := rows.Scan(
&i.ID,
&i.Trigger,
&i.Status,
&i.StartedAt,
&i.FinishedAt,
&i.Location,
&i.Bytes,
&i.Error,
&i.MusicFiles,
&i.MusicBytes,
&i.MusicNote,
); err != nil {
return nil, err
}
items = append(items, i)
}
if err := rows.Err(); err != nil {
return nil, err
}
return items, nil
}
const pendingBackupMusic = `-- name: PendingBackupMusic :many
SELECT t.id, t.file_path, t.file_size,
m.size AS sent_size, m.mtime AS sent_mtime, m.error AS sent_error
FROM tracks t
LEFT JOIN backup_music_files m ON m.track_id = t.id
WHERE t.missing_since IS NULL
AND t.file_path > $1::text
AND (m.track_id IS NULL OR m.size <> t.file_size OR t.updated_at > m.checked_at
OR (m.error <> '' AND m.checked_at < (SELECT updated_at FROM backup_settings)))
ORDER BY t.file_path
LIMIT $2
`
type PendingBackupMusicParams struct {
AfterPath string
MaxRows int32
}
type PendingBackupMusicRow struct {
ID pgtype.UUID
FilePath string
FileSize int64
SentSize *int64
SentMtime pgtype.Timestamptz
SentError *string
}
// Library files the backup may not hold as they are now, in path order from
// after_path, so a night that runs out of time carries on where it stopped:
// never sent, a different size, touched by a scan since they were last
// checked, or refused before the settings were last saved (a fixed key gets
// another go). The caller compares the file's own size and mtime before
// sending.
func (q *Queries) PendingBackupMusic(ctx context.Context, arg PendingBackupMusicParams) ([]PendingBackupMusicRow, error) {
rows, err := q.db.Query(ctx, pendingBackupMusic, arg.AfterPath, arg.MaxRows)
if err != nil {
return nil, err
}
defer rows.Close()
var items []PendingBackupMusicRow
for rows.Next() {
var i PendingBackupMusicRow
if err := rows.Scan(
&i.ID,
&i.FilePath,
&i.FileSize,
&i.SentSize,
&i.SentMtime,
&i.SentError,
); err != nil {
return nil, err
}
items = append(items, i)
}
if err := rows.Err(); err != nil {
return nil, err
}
return items, nil
}
const pruneBackupRuns = `-- name: PruneBackupRuns :execrows
DELETE FROM backup_runs WHERE status <> 'running' AND started_at < $1
`
func (q *Queries) PruneBackupRuns(ctx context.Context, startedAt pgtype.Timestamptz) (int64, error) {
result, err := q.db.Exec(ctx, pruneBackupRuns, startedAt)
if err != nil {
return 0, err
}
return result.RowsAffected(), nil
}
const recordBackupMusic = `-- name: RecordBackupMusic :exec
INSERT INTO backup_music_files (track_id, size, mtime, location, error)
VALUES ($1, $2, $3, $4, $5)
ON CONFLICT (track_id) DO UPDATE
SET size = EXCLUDED.size, mtime = EXCLUDED.mtime, location = EXCLUDED.location,
error = EXCLUDED.error, checked_at = now()
`
type RecordBackupMusicParams struct {
TrackID pgtype.UUID
Size int64
Mtime pgtype.Timestamptz
Location string
Error string
}
func (q *Queries) RecordBackupMusic(ctx context.Context, arg RecordBackupMusicParams) error {
_, err := q.db.Exec(ctx, recordBackupMusic,
arg.TrackID,
arg.Size,
arg.Mtime,
arg.Location,
arg.Error,
)
return err
}
const refusedBackupMusic = `-- name: RefusedBackupMusic :many
SELECT t.file_path, m.error, m.checked_at
FROM backup_music_files m
JOIN tracks t ON t.id = m.track_id
WHERE m.error <> '' AND m.size = t.file_size AND t.updated_at <= m.checked_at
ORDER BY m.checked_at DESC
LIMIT $1
`
type RefusedBackupMusicRow struct {
FilePath string
Error string
CheckedAt pgtype.Timestamptz
}
func (q *Queries) RefusedBackupMusic(ctx context.Context, limit int32) ([]RefusedBackupMusicRow, error) {
rows, err := q.db.Query(ctx, refusedBackupMusic, limit)
if err != nil {
return nil, err
}
defer rows.Close()
var items []RefusedBackupMusicRow
for rows.Next() {
var i RefusedBackupMusicRow
if err := rows.Scan(&i.FilePath, &i.Error, &i.CheckedAt); err != nil {
return nil, err
}
items = append(items, i)
}
if err := rows.Err(); err != nil {
return nil, err
}
return items, nil
}
const saveBackupSettings = `-- name: SaveBackupSettings :exec
UPDATE backup_settings
SET enabled = $1,
destination = $2,
folder = $3,
endpoint = $4,
bucket = $5,
access_key = $6,
secret_key = $7,
run_hour = $8,
upload_limit_kib = $9,
include_music = $10,
updated_at = now()
WHERE id = true
`
type SaveBackupSettingsParams struct {
Enabled bool
Destination string
Folder string
Endpoint string
Bucket string
AccessKey string
SecretKey string
RunHour int16
UploadLimitKib int32
IncludeMusic bool
}
func (q *Queries) SaveBackupSettings(ctx context.Context, arg SaveBackupSettingsParams) error {
_, err := q.db.Exec(ctx, saveBackupSettings,
arg.Enabled,
arg.Destination,
arg.Folder,
arg.Endpoint,
arg.Bucket,
arg.AccessKey,
arg.SecretKey,
arg.RunHour,
arg.UploadLimitKib,
arg.IncludeMusic,
)
return err
}
const startBackupRun = `-- name: StartBackupRun :one
INSERT INTO backup_runs (trigger) VALUES ($1)
RETURNING id, started_at
`
type StartBackupRunRow struct {
ID pgtype.UUID
StartedAt pgtype.Timestamptz
}
func (q *Queries) StartBackupRun(ctx context.Context, trigger string) (StartBackupRunRow, error) {
row := q.db.QueryRow(ctx, startBackupRun, trigger)
var i StartBackupRunRow
err := row.Scan(&i.ID, &i.StartedAt)
return i, err
}
+286
View File
@@ -0,0 +1,286 @@
// Code generated by sqlc. DO NOT EDIT.
// versions:
// sqlc v1.31.1
// source: genre_mixes.sql
package dbq
import (
"context"
"github.com/jackc/pgx/v5/pgtype"
)
const listGenreMixCandidates = `-- name: ListGenreMixCandidates :many
WITH library_genres AS (
SELECT lower(trim(g.g)) AS genre_key,
min(trim(g.g))::text AS label,
COUNT(DISTINCT t.id) AS track_count,
array_agg(DISTINCT t.id)::uuid[] AS track_ids
FROM tracks t
CROSS JOIN LATERAL regexp_split_to_table(coalesce(t.genre, ''), '[;,]') AS g(g)
WHERE t.missing_since IS NULL
AND trim(g.g) <> ''
GROUP BY lower(trim(g.g))
HAVING COUNT(DISTINCT t.id) >= 25
AND COUNT(DISTINCT t.artist_id) >= 3
),
taste AS (
SELECT lower(trim(tpt.tag)) AS genre_key, SUM(tpt.weight) AS weight
FROM taste_profile_tags tpt
WHERE tpt.user_id = $1
GROUP BY lower(trim(tpt.tag))
HAVING SUM(tpt.weight) > 0
),
listened AS (
SELECT t.id
FROM play_events pe
JOIN tracks t ON t.id = pe.track_id
WHERE pe.user_id = $1 AND pe.was_skipped = false
UNION ALL
SELECT gl.track_id
FROM general_likes gl
WHERE gl.user_id = $1
),
history AS (
SELECT lower(trim(g.g)) AS genre_key, COUNT(*) AS plays
FROM listened l
JOIN tracks t ON t.id = l.id
CROSS JOIN LATERAL regexp_split_to_table(coalesce(t.genre, ''), '[;,]') AS g(g)
WHERE trim(g.g) <> ''
GROUP BY lower(trim(g.g))
)
SELECT lg.genre_key::text AS genre_key,
lg.label AS label,
(CASE WHEN ta.genre_key IS NOT NULL THEN 1
WHEN h.genre_key IS NOT NULL THEN 2
ELSE 3 END)::int AS tier,
(CASE WHEN ta.genre_key IS NOT NULL THEN ta.weight
WHEN h.genre_key IS NOT NULL THEN h.plays::float8
ELSE lg.track_count::float8 END)::float8 AS score,
lg.track_ids
FROM library_genres lg
LEFT JOIN taste ta ON ta.genre_key = lg.genre_key
LEFT JOIN history h ON h.genre_key = lg.genre_key
ORDER BY tier, score DESC, lg.genre_key
LIMIT 60
`
type ListGenreMixCandidatesRow struct {
GenreKey string
Label string
Tier int32
Score float64
TrackIds []pgtype.UUID
}
// Candidate queries for the genre mixes: one daily system playlist per genre
// the user listens to. Genres live inline on tracks.genre as a delimited
// string and are split on [;,] the same way the Discover and radio tag arms
// split them. Matching is case-insensitive, so the ID3 "Post-Punk" and the
// folksonomy "post-punk" in the taste profile count as one genre.
// Library genres big enough to carry a mix (>= 25 playable tracks across
// >= 3 artists), each tagged with how close it is to the user's listening:
//
// tier 1: a positive taste-profile genre, scored by its taste weight
// tier 2: a genre of tracks the user has played through or liked, scored
// by how many — catches a profile that has not been built yet or
// has decayed past its window
// tier 3: any other library genre, scored by its size
//
// track_ids lets the caller tell how much music two genres share: a genre
// whose songs mostly sit in genres already picked adds no new channel, and a
// broad genre ("Rock") whose songs mostly carry narrower ones ("Pop Punk",
// "Ska Punk") gives way to them.
// $1 user_id.
func (q *Queries) ListGenreMixCandidates(ctx context.Context, userID pgtype.UUID) ([]ListGenreMixCandidatesRow, error) {
rows, err := q.db.Query(ctx, listGenreMixCandidates, userID)
if err != nil {
return nil, err
}
defer rows.Close()
var items []ListGenreMixCandidatesRow
for rows.Next() {
var i ListGenreMixCandidatesRow
if err := rows.Scan(
&i.GenreKey,
&i.Label,
&i.Tier,
&i.Score,
&i.TrackIds,
); err != nil {
return nil, err
}
items = append(items, i)
}
if err := rows.Err(); err != nil {
return nil, err
}
return items, nil
}
const listGenreMixTracks = `-- name: ListGenreMixTracks :many
WITH play_counts AS (
SELECT pe.track_id, COUNT(*) AS c
FROM play_events pe
WHERE pe.user_id = $1 AND pe.was_skipped = false
GROUP BY pe.track_id
),
skip_counts AS (
SELECT pe.track_id, COUNT(*) AS c
FROM play_events pe
WHERE pe.user_id = $1 AND pe.was_skipped = true
GROUP BY pe.track_id
),
artist_plays AS (
SELECT t.artist_id,
COUNT(*) FILTER (WHERE NOT pe.was_skipped) AS plays,
COUNT(*) FILTER (WHERE pe.was_skipped) AS skips
FROM play_events pe
JOIN tracks t ON t.id = pe.track_id
WHERE pe.user_id = $1
GROUP BY t.artist_id
),
artist_likes AS (
SELECT t.artist_id, COUNT(*) AS likes
FROM general_likes gl
JOIN tracks t ON t.id = gl.track_id
WHERE gl.user_id = $1
GROUP BY t.artist_id
),
own_artists AS (
SELECT ap.artist_id FROM artist_plays ap WHERE ap.plays > 0
UNION
SELECT al.artist_id FROM artist_likes al
),
artist_sim AS (
SELECT asim.artist_b_id AS artist_id,
MAX(LEAST(GREATEST(asim.score, 0), 1)) AS sim
FROM artist_similarity asim
JOIN own_artists oa ON oa.artist_id = asim.artist_a_id
GROUP BY asim.artist_b_id
),
eligible AS (
SELECT t.id, t.album_id, t.artist_id,
(COALESCE(pc.c, 0) > 0 OR gl.track_id IS NOT NULL) AS familiar,
CASE
WHEN COALESCE(pc.c, 0) > 0 OR gl.track_id IS NOT NULL THEN
(1 + ln(1 + COALESCE(pc.c, 0)::float8)
+ CASE WHEN gl.track_id IS NOT NULL THEN 1 ELSE 0 END)
* ((COALESCE(pc.c, 0) + 1)::float8 / (COALESCE(pc.c, 0) + COALESCE(sc.c, 0) + 1))
ELSE
(1 + ln(1 + COALESCE(ap.plays, 0)::float8 + 2 * COALESCE(al.likes, 0)::float8)
+ 2 * COALESCE(asm.sim, 0))
* ((COALESCE(ap.plays, 0) + 1)::float8 / (COALESCE(ap.plays, 0) + COALESCE(ap.skips, 0) + 1))
END::float8 AS weight,
-- u in (0, 1): the first 32 bits of md5(track, date), shifted off 0.
((('x' || substr(md5(t.id::text || $2::text), 1, 8))::bit(32)::bigint)::float8 + 1)
/ 4294967297.0 AS u
FROM tracks t
LEFT JOIN play_counts pc ON pc.track_id = t.id
LEFT JOIN skip_counts sc ON sc.track_id = t.id
LEFT JOIN general_likes gl ON gl.user_id = $1 AND gl.track_id = t.id
LEFT JOIN artist_plays ap ON ap.artist_id = t.artist_id
LEFT JOIN artist_likes al ON al.artist_id = t.artist_id
LEFT JOIN artist_sim asm ON asm.artist_id = t.artist_id
WHERE t.missing_since IS NULL -- never offer a file that is gone
AND NOT (COALESCE(sc.c, 0) >= 2 AND COALESCE(sc.c, 0) > COALESCE(pc.c, 0))
AND EXISTS (
SELECT 1
FROM regexp_split_to_table(coalesce(t.genre, ''), '[;,]') AS g(g)
WHERE lower(trim(g.g)) = $3::text
)
AND NOT EXISTS (
SELECT 1
FROM regexp_split_to_table(coalesce(t.genre, ''), '[;,]') AS g(g)
WHERE lower(trim(g.g)) = ANY($4::text[])
)
AND NOT EXISTS (
SELECT 1 FROM quarantine q
WHERE q.user_id = $1 AND q.track_id = t.id
)
),
ranked AS (
SELECT e.id, e.album_id, e.artist_id, e.familiar,
-ln(e.u) / e.weight AS draw,
row_number() OVER (PARTITION BY e.familiar ORDER BY -ln(e.u) / e.weight, e.id) AS rn
FROM eligible e
)
SELECT r.id, r.album_id, r.artist_id, r.familiar::bool AS familiar
FROM ranked r
WHERE r.rn <= 200
ORDER BY r.familiar DESC, r.draw, r.id
`
type ListGenreMixTracksParams struct {
UserID pgtype.UUID
DateSeed string
GenreKey string
ExcludeGenres []string
}
type ListGenreMixTracksRow struct {
ID pgtype.UUID
AlbumID pgtype.UUID
ArtistID pgtype.UUID
Familiar bool
}
// Playable tracks carrying one genre, split into the two halves a genre mix
// blends: familiar (the user has played it through or liked it) and unheard.
// Up to 200 of each, drawn by a weighted lottery seeded with the date, so the
// mix changes day to day, holds still within a day, and leans toward what the
// user likes without becoming the same 50 songs every day.
//
// The lottery is the Efraimidis-Spirakis draw: each track gets
// -ln(u) / weight with u uniform in (0, 1) from md5(track, date), and the
// smallest keys win. A track of weight 3 is three times as likely as one of
// weight 1 to come before it, and every track keeps some chance.
//
// Familiar weight: 1 + ln(1 + plays through) + 1 if liked, times the share of
// its plays that were not skips. A song played 40 times weighs about 4.7, one
// played once about 1.7.
//
// Unheard weight: 1 + ln(1 + the artist's plays through + 2 × its liked
// tracks) + 2 × how similar the artist is to the user's own artists (the best
// artist_similarity score from any artist they have played through or liked,
// 0 to 1), times the share of the artist's plays that were not skips. An
// artist the user has never met weighs 1, so new artists still come up; an
// artist they keep skipping weighs well under 1.
//
// A track the user keeps skipping stays out: twice-skipped and skipped more
// often than played through. The second half of that test keeps a favourite
// with a couple of stray skips in.
//
// exclude_genres keeps each song in one mix: a track carrying a genre that
// already has a mix today is left to that mix.
func (q *Queries) ListGenreMixTracks(ctx context.Context, arg ListGenreMixTracksParams) ([]ListGenreMixTracksRow, error) {
rows, err := q.db.Query(ctx, listGenreMixTracks,
arg.UserID,
arg.DateSeed,
arg.GenreKey,
arg.ExcludeGenres,
)
if err != nil {
return nil, err
}
defer rows.Close()
var items []ListGenreMixTracksRow
for rows.Next() {
var i ListGenreMixTracksRow
if err := rows.Scan(
&i.ID,
&i.AlbumID,
&i.ArtistID,
&i.Familiar,
); err != nil {
return nil, err
}
items = append(items, i)
}
if err := rows.Err(); err != nil {
return nil, err
}
return items, nil
}
+57 -18
View File
@@ -11,21 +11,58 @@ import (
"github.com/jackc/pgx/v5/pgtype"
)
const deleteOldLibraryChanges = `-- name: DeleteOldLibraryChanges :execrows
DELETE FROM library_changes WHERE changed_at < $1
const deleteOldLibraryDeletes = `-- name: DeleteOldLibraryDeletes :one
WITH gone AS (
DELETE FROM library_changes WHERE op = 'delete' AND changed_at < $1 RETURNING id
), mark AS (
UPDATE library_changes_trimmed
SET through = GREATEST(through, (SELECT COALESCE(max(id), 0) FROM gone))
)
SELECT count(*)::BIGINT FROM gone
`
// Removes library_changes rows older than the given cutoff. Run from
// the sync compactor goroutine on a daily tick. Returns the number of
// rows deleted so the worker can log meaningful progress.
func (q *Queries) DeleteOldLibraryChanges(ctx context.Context, changedAt pgtype.Timestamptz) (int64, error) {
result, err := q.db.Exec(ctx, deleteOldLibraryChanges, changedAt)
// Removes deletes older than the cutoff and raises the trimmed mark past the
// highest one, so an app whose cursor is below it starts over. Returns how
// many went.
func (q *Queries) DeleteOldLibraryDeletes(ctx context.Context, changedAt pgtype.Timestamptz) (int64, error) {
row := q.db.QueryRow(ctx, deleteOldLibraryDeletes, changedAt)
var column_1 int64
err := row.Scan(&column_1)
return column_1, err
}
const deleteSupersededLibraryChanges = `-- name: DeleteSupersededLibraryChanges :execrows
DELETE FROM library_changes old
WHERE EXISTS (SELECT 1 FROM library_changes newer
WHERE newer.entity_type = old.entity_type
AND newer.entity_id = old.entity_id
AND newer.id > old.id)
`
// Removes every row a newer row for the same entity has superseded. An app
// reading from any cursor still meets the newer row, so it loses nothing.
func (q *Queries) DeleteSupersededLibraryChanges(ctx context.Context) (int64, error) {
result, err := q.db.Exec(ctx, deleteSupersededLibraryChanges)
if err != nil {
return 0, err
}
return result.RowsAffected(), nil
}
const getLastIssuedLibraryChangeCursor = `-- name: GetLastIssuedLibraryChangeCursor :one
SELECT COALESCE(pg_sequence_last_value(pg_get_serial_sequence('library_changes', 'id')::regclass), 0)::BIGINT
`
// The highest cursor this server has handed out, even once the compactor has
// trimmed the rows: the id sequence's last value. A restored server's
// sequence stands at the backup's last change.
func (q *Queries) GetLastIssuedLibraryChangeCursor(ctx context.Context) (int64, error) {
row := q.db.QueryRow(ctx, getLastIssuedLibraryChangeCursor)
var column_1 int64
err := row.Scan(&column_1)
return column_1, err
}
const getLibraryChangesSince = `-- name: GetLibraryChangesSince :many
SELECT id, entity_type, entity_id, op, changed_at
FROM library_changes
@@ -65,6 +102,19 @@ func (q *Queries) GetLibraryChangesSince(ctx context.Context, arg GetLibraryChan
return items, nil
}
const getLibraryChangesTrimmedThrough = `-- name: GetLibraryChangesTrimmedThrough :one
SELECT through FROM library_changes_trimmed
`
// The highest id of a delete the compactor has dropped. A cursor below it may
// have missed that delete.
func (q *Queries) GetLibraryChangesTrimmedThrough(ctx context.Context) (int64, error) {
row := q.db.QueryRow(ctx, getLibraryChangesTrimmedThrough)
var through int64
err := row.Scan(&through)
return through, err
}
const getMaxLibraryChangeCursor = `-- name: GetMaxLibraryChangeCursor :one
SELECT COALESCE(MAX(id), 0)::BIGINT FROM library_changes
`
@@ -76,17 +126,6 @@ func (q *Queries) GetMaxLibraryChangeCursor(ctx context.Context) (int64, error)
return column_1, err
}
const getMinLibraryChangeCursor = `-- name: GetMinLibraryChangeCursor :one
SELECT COALESCE(MIN(id), 0)::BIGINT FROM library_changes
`
func (q *Queries) GetMinLibraryChangeCursor(ctx context.Context) (int64, error) {
row := q.db.QueryRow(ctx, getMinLibraryChangeCursor)
var column_1 int64
err := row.Scan(&column_1)
return column_1, err
}
const insertLibraryChange = `-- name: InsertLibraryChange :exec
INSERT INTO library_changes (entity_type, entity_id, op)
VALUES ($1, $2, $3)
+43
View File
@@ -305,6 +305,44 @@ type AuditLog struct {
CreatedAt pgtype.Timestamptz
}
type BackupMusicFile struct {
TrackID pgtype.UUID
Size int64
Mtime pgtype.Timestamptz
Location string
CheckedAt pgtype.Timestamptz
Error string
}
type BackupRun struct {
ID pgtype.UUID
Trigger string
Status string
StartedAt pgtype.Timestamptz
FinishedAt pgtype.Timestamptz
Location string
Bytes int64
Error string
MusicFiles int32
MusicBytes int64
MusicNote string
}
type BackupSetting struct {
ID bool
Enabled bool
Destination string
Folder string
Endpoint string
Bucket string
AccessKey string
SecretKey string
RunHour int16
UploadLimitKib int32
UpdatedAt pgtype.Timestamptz
IncludeMusic bool
}
type CandidateArtistTag struct {
CandidateMbid string
Tag string
@@ -506,6 +544,11 @@ type LibraryChange struct {
ChangedAt pgtype.Timestamptz
}
type LibraryChangesTrimmed struct {
ID bool
Through int64
}
type LibrarySetting struct {
ID bool
ScanPaths []string
@@ -0,0 +1,2 @@
DROP TABLE IF EXISTS backup_runs;
DROP TABLE IF EXISTS backup_settings;
@@ -0,0 +1,44 @@
-- 0099_backups.up.sql — Minstrel backing itself up.
--
-- backup_settings one row: whether the nightly backup is on, where it goes
-- (a folder, or an S3 bucket such as a Fabled Archives
-- target), and the hour it runs, in UTC. Off until an
-- admin switches it on.
-- backup_runs one row per attempt: when, what started it, how it ended,
-- where the file went, its size, and the error in words.
CREATE TABLE backup_settings (
id boolean PRIMARY KEY DEFAULT true CONSTRAINT backup_settings_one_row CHECK (id),
enabled boolean NOT NULL DEFAULT false,
destination text NOT NULL DEFAULT 's3'
CONSTRAINT backup_settings_destination_check CHECK (destination IN ('folder', 's3')),
folder text NOT NULL DEFAULT '',
endpoint text NOT NULL DEFAULT '',
bucket text NOT NULL DEFAULT '',
access_key text NOT NULL DEFAULT '',
-- Sealed under the stream secret, as every stored credential is.
secret_key text NOT NULL DEFAULT '',
run_hour smallint NOT NULL DEFAULT 3
CONSTRAINT backup_settings_run_hour_check CHECK (run_hour BETWEEN 0 AND 23),
-- The most an upload to the bucket may send, in KiB a second; 0 is no limit.
upload_limit_kib integer NOT NULL DEFAULT 4096
CONSTRAINT backup_settings_upload_limit_check CHECK (upload_limit_kib >= 0),
updated_at timestamptz NOT NULL DEFAULT now()
);
INSERT INTO backup_settings DEFAULT VALUES;
CREATE TABLE backup_runs (
id uuid PRIMARY KEY DEFAULT gen_random_uuid(),
trigger text NOT NULL
CONSTRAINT backup_runs_trigger_check CHECK (trigger IN ('schedule', 'manual')),
status text NOT NULL DEFAULT 'running'
CONSTRAINT backup_runs_status_check CHECK (status IN ('running', 'succeeded', 'failed')),
started_at timestamptz NOT NULL DEFAULT now(),
finished_at timestamptz,
-- Where the file went: the object key, or the file's path in the folder.
location text NOT NULL DEFAULT '',
bytes bigint NOT NULL DEFAULT 0,
error text NOT NULL DEFAULT ''
);
CREATE INDEX backup_runs_started_at_idx ON backup_runs (started_at DESC);
@@ -0,0 +1,6 @@
DROP TABLE IF EXISTS backup_music_files;
ALTER TABLE backup_runs
DROP COLUMN IF EXISTS music_note,
DROP COLUMN IF EXISTS music_bytes,
DROP COLUMN IF EXISTS music_files;
ALTER TABLE backup_settings DROP COLUMN IF EXISTS include_music;
@@ -0,0 +1,36 @@
-- 0100_backup_music.up.sql — the music files in Minstrel's own backup.
--
-- Until now a backup held the database only. With include_music on, each
-- library file is also sent once to music/<path> in the same destination,
-- and sent again only when it changes.
--
-- backup_settings.include_music off until an admin switches it on.
-- backup_music_files one row per track sent (or refused): the
-- file's size and mtime when it was sent, so
-- a later night sends only what is new or
-- changed. A track the scan has touched since
-- checked_at is looked at again; it goes up
-- again only if the file itself changed, so a
-- full rescan doesn't resend the library. A
-- refusal is kept with its reason and tried
-- again once the file changes.
-- backup_runs.music_* how much a run sent, and why it stopped.
ALTER TABLE backup_settings ADD COLUMN include_music boolean NOT NULL DEFAULT false;
ALTER TABLE backup_runs
ADD COLUMN music_files integer NOT NULL DEFAULT 0,
ADD COLUMN music_bytes bigint NOT NULL DEFAULT 0,
ADD COLUMN music_note text NOT NULL DEFAULT '';
CREATE TABLE backup_music_files (
track_id uuid PRIMARY KEY REFERENCES tracks (id) ON DELETE CASCADE,
-- The file as it was sent.
size bigint NOT NULL,
mtime timestamptz NOT NULL,
-- Where it went: the object key, or the file's path in the folder.
location text NOT NULL,
-- When the file was last sent or found unchanged.
checked_at timestamptz NOT NULL DEFAULT now(),
-- Empty when it was sent; otherwise why the destination refused it.
error text NOT NULL DEFAULT ''
);
@@ -0,0 +1,2 @@
DROP TABLE IF EXISTS library_changes_trimmed;
DROP INDEX IF EXISTS idx_library_changes_entity;
@@ -0,0 +1,56 @@
-- 0101_library_changes_keep_latest.up.sql — the change feed keeps every
-- entity's latest change, so an app syncing from nothing gets the whole
-- library.
--
-- Until now the compactor dropped every row older than 30 days. An app
-- syncing from cursor 0 (a fresh install, cleared data, or a reset after a
-- 410) builds its library from the feed, so it got only what had changed in
-- the last 30 days. From here the compactor drops a row only once a newer row
-- for the same entity exists, and drops a delete only once it is 30 days old.
--
-- idx_library_changes_entity finds an entity's rows, newest first, for
-- the compactor and the fill below.
-- library_changes_trimmed one row: the highest id of a delete the
-- compactor has dropped. An app whose cursor
-- is below it may have missed that delete, so
-- it is told to start over (410). It starts
-- where the old compactor left off: everything
-- below the oldest remaining row is gone.
--
-- Then every artist, album and track whose latest row is not an upsert, or
-- which has no row at all (the old compactor dropped it), gets one, artists
-- first so an album never arrives before its artist. Likes and playlists are
-- left as they are: the apps fetch those from their own endpoints.
CREATE INDEX idx_library_changes_entity ON library_changes (entity_type, entity_id, id DESC);
CREATE TABLE library_changes_trimmed (
id boolean PRIMARY KEY DEFAULT true CONSTRAINT library_changes_trimmed_one_row CHECK (id),
through bigint NOT NULL DEFAULT 0
);
INSERT INTO library_changes_trimmed (through)
SELECT COALESCE(min(id), 1) - 1 FROM library_changes;
INSERT INTO library_changes (entity_type, entity_id, op)
SELECT 'artist', a.id::text, 'upsert'
FROM artists a
WHERE COALESCE((SELECT c.op FROM library_changes c
WHERE c.entity_type = 'artist' AND c.entity_id = a.id::text
ORDER BY c.id DESC LIMIT 1), 'delete') <> 'upsert'
ORDER BY a.id;
INSERT INTO library_changes (entity_type, entity_id, op)
SELECT 'album', al.id::text, 'upsert'
FROM albums al
WHERE COALESCE((SELECT c.op FROM library_changes c
WHERE c.entity_type = 'album' AND c.entity_id = al.id::text
ORDER BY c.id DESC LIMIT 1), 'delete') <> 'upsert'
ORDER BY al.id;
INSERT INTO library_changes (entity_type, entity_id, op)
SELECT 'track', t.id::text, 'upsert'
FROM tracks t
WHERE COALESCE((SELECT c.op FROM library_changes c
WHERE c.entity_type = 'track' AND c.entity_id = t.id::text
ORDER BY c.id DESC LIMIT 1), 'delete') <> 'upsert'
ORDER BY t.id;
@@ -0,0 +1,37 @@
-- Restores the 0028 constraint set. Genre-mix rows would violate it, so they
-- go first.
DELETE FROM playlists WHERE system_variant = 'genre_mix';
ALTER TABLE playlists DROP CONSTRAINT IF EXISTS playlists_kind_variant_consistent;
ALTER TABLE playlists
ADD CONSTRAINT playlists_kind_variant_consistent CHECK (
(kind = 'user' AND system_variant IS NULL)
OR
(kind = 'system' AND system_variant IN (
'for_you', 'songs_like_artist', 'discover',
'deep_cuts', 'rediscover', 'new_for_you',
'on_this_day', 'first_listens'
))
);
ALTER TABLE playlists DROP CONSTRAINT IF EXISTS playlists_seed_consistent;
ALTER TABLE playlists
ADD CONSTRAINT playlists_seed_consistent CHECK (
(system_variant IS NULL AND seed_artist_id IS NULL)
OR
(system_variant = 'for_you' AND seed_artist_id IS NULL)
OR
(system_variant = 'discover' AND seed_artist_id IS NULL)
OR
(system_variant = 'deep_cuts' AND seed_artist_id IS NULL)
OR
(system_variant = 'rediscover' AND seed_artist_id IS NULL)
OR
(system_variant = 'new_for_you' AND seed_artist_id IS NULL)
OR
(system_variant = 'on_this_day' AND seed_artist_id IS NULL)
OR
(system_variant = 'first_listens' AND seed_artist_id IS NULL)
OR
(system_variant = 'songs_like_artist' AND seed_artist_id IS NOT NULL)
);
@@ -0,0 +1,43 @@
-- Genre mixes: one daily system playlist per genre the user listens to
-- ("Post-Punk mix", "Jazz mix", ...). Several per user, like
-- songs_like_artist, but seedless: the genre lives in the playlist name and
-- seed_artist_id is NULL.
--
-- Postgres has no ALTER CONSTRAINT for CHECK, so both constraints are
-- dropped and re-added with 'genre_mix' in the accepted set.
ALTER TABLE playlists DROP CONSTRAINT IF EXISTS playlists_kind_variant_consistent;
ALTER TABLE playlists
ADD CONSTRAINT playlists_kind_variant_consistent CHECK (
(kind = 'user' AND system_variant IS NULL)
OR
(kind = 'system' AND system_variant IN (
'for_you', 'songs_like_artist', 'discover',
'deep_cuts', 'rediscover', 'new_for_you',
'on_this_day', 'first_listens', 'genre_mix'
))
);
ALTER TABLE playlists DROP CONSTRAINT IF EXISTS playlists_seed_consistent;
ALTER TABLE playlists
ADD CONSTRAINT playlists_seed_consistent CHECK (
(system_variant IS NULL AND seed_artist_id IS NULL)
OR
(system_variant = 'for_you' AND seed_artist_id IS NULL)
OR
(system_variant = 'discover' AND seed_artist_id IS NULL)
OR
(system_variant = 'deep_cuts' AND seed_artist_id IS NULL)
OR
(system_variant = 'rediscover' AND seed_artist_id IS NULL)
OR
(system_variant = 'new_for_you' AND seed_artist_id IS NULL)
OR
(system_variant = 'on_this_day' AND seed_artist_id IS NULL)
OR
(system_variant = 'first_listens' AND seed_artist_id IS NULL)
OR
(system_variant = 'genre_mix' AND seed_artist_id IS NULL)
OR
(system_variant = 'songs_like_artist' AND seed_artist_id IS NOT NULL)
);
+110
View File
@@ -0,0 +1,110 @@
-- Minstrel's own backup: its settings and its runs.
-- name: GetBackupSettings :one
SELECT * FROM backup_settings WHERE id = true;
-- name: SaveBackupSettings :exec
UPDATE backup_settings
SET enabled = $1,
destination = $2,
folder = $3,
endpoint = $4,
bucket = $5,
access_key = $6,
secret_key = $7,
run_hour = $8,
upload_limit_kib = $9,
include_music = $10,
updated_at = now()
WHERE id = true;
-- name: StartBackupRun :one
INSERT INTO backup_runs (trigger) VALUES ($1)
RETURNING id, started_at;
-- name: FinishBackupRun :exec
UPDATE backup_runs
SET status = $2,
finished_at = now(),
location = $3,
bytes = $4,
error = $5,
music_files = $6,
music_bytes = $7,
music_note = $8
WHERE id = $1;
-- name: ListBackupRuns :many
SELECT id, trigger, status, started_at, finished_at, location, bytes, error,
music_files, music_bytes, music_note
FROM backup_runs
ORDER BY started_at DESC
LIMIT $1;
-- name: BackupRunTimes :one
-- The times the verdict turns on. A run that never happened is the epoch,
-- so the row always exists. failing_since is the first failure after the last
-- success, and the epoch when the latest finished run succeeded.
SELECT
COALESCE((SELECT max(started_at) FROM backup_runs WHERE status = 'succeeded'),
'epoch'::timestamptz)::timestamptz AS last_success,
COALESCE((SELECT max(started_at) FROM backup_runs),
'epoch'::timestamptz)::timestamptz AS last_attempt,
COALESCE((SELECT min(started_at) FROM backup_runs
WHERE status = 'failed'
AND started_at > COALESCE((SELECT max(started_at) FROM backup_runs WHERE status = 'succeeded'),
'epoch'::timestamptz)),
'epoch'::timestamptz)::timestamptz AS failing_since;
-- name: FailUnfinishedBackupRuns :execrows
UPDATE backup_runs
SET status = 'failed', finished_at = now(),
error = 'The server stopped while the backup was running.'
WHERE status = 'running';
-- name: PruneBackupRuns :execrows
DELETE FROM backup_runs WHERE status <> 'running' AND started_at < $1;
-- name: PendingBackupMusic :many
-- Library files the backup may not hold as they are now, in path order from
-- after_path, so a night that runs out of time carries on where it stopped:
-- never sent, a different size, touched by a scan since they were last
-- checked, or refused before the settings were last saved (a fixed key gets
-- another go). The caller compares the file's own size and mtime before
-- sending.
SELECT t.id, t.file_path, t.file_size,
m.size AS sent_size, m.mtime AS sent_mtime, m.error AS sent_error
FROM tracks t
LEFT JOIN backup_music_files m ON m.track_id = t.id
WHERE t.missing_since IS NULL
AND t.file_path > sqlc.arg(after_path)::text
AND (m.track_id IS NULL OR m.size <> t.file_size OR t.updated_at > m.checked_at
OR (m.error <> '' AND m.checked_at < (SELECT updated_at FROM backup_settings)))
ORDER BY t.file_path
LIMIT sqlc.arg(max_rows);
-- name: RecordBackupMusic :exec
INSERT INTO backup_music_files (track_id, size, mtime, location, error)
VALUES ($1, $2, $3, $4, $5)
ON CONFLICT (track_id) DO UPDATE
SET size = EXCLUDED.size, mtime = EXCLUDED.mtime, location = EXCLUDED.location,
error = EXCLUDED.error, checked_at = now();
-- name: BackupMusicProgress :one
-- How much of the library the backup holds as it is now.
SELECT count(*)::bigint AS total_files,
COALESCE(sum(t.file_size), 0)::bigint AS total_bytes,
count(*) FILTER (WHERE m.error = '' AND m.size = t.file_size AND t.updated_at <= m.checked_at)::bigint AS sent_files,
COALESCE(sum(t.file_size) FILTER (WHERE m.error = '' AND m.size = t.file_size AND t.updated_at <= m.checked_at), 0)::bigint AS sent_bytes,
count(*) FILTER (WHERE m.error <> '' AND m.size = t.file_size AND t.updated_at <= m.checked_at)::bigint AS refused_files
FROM tracks t
LEFT JOIN backup_music_files m ON m.track_id = t.id
WHERE t.missing_since IS NULL;
-- name: RefusedBackupMusic :many
SELECT t.file_path, m.error, m.checked_at
FROM backup_music_files m
JOIN tracks t ON t.id = m.track_id
WHERE m.error <> '' AND m.size = t.file_size AND t.updated_at <= m.checked_at
ORDER BY m.checked_at DESC
LIMIT $1;

Some files were not shown because too many files have changed in this diff Show More