Commit Graph
6 Commits
Author SHA1 Message Date
bvandeusenandClaude Opus 5.5 9d324ab2d0 feat: Minstrel backs itself up nightly to a folder or an S3 bucket, off until an admin switches it on (#6773)
release / integration (push) Failing after 3m29s
release / web (push) Failing after 15s
release / govulncheck (push) Successful in 16s
release / go (push) Failing after 41s
release / android (push) Successful in 4m7s
release / Attach APK to the Release (tag releases only) (push) Canceled after 0s
release / Build + push container image (push) Canceled after 0s
release / Verify release artifacts (tag releases only) (push) Canceled after 0s
release / Build signed APK (push) Canceled after 5m34s
Admin › Backups shows a verdict first (off, not set up, waiting, backed up,
overdue, failing since when), Back up now, Test destination and the latest
runs, with the settings beside them.

- backup.Service: one run at a time, bounded at 30 minutes; the schedule
  ticks every 5 minutes, catches up a slot missed while the server was down,
  and retries a failure after an hour and then after as long as it has been
  failing. Runs are recorded in backup_runs, pruned after 90 days; a run cut
  off by a shutdown is marked failed at the next start.
- A folder destination writes database/minstrel-<UTC stamp>.tar and refuses
  a folder that isn't there. An S3 destination stages the file, hashes it,
  and uploads it under database/ through a throttled reader (4 MiB/s by
  default, 0 for no limit).
- Test destination checks the form as typed and only lists.
- The bucket secret is sealed like every stored credential; backup_settings
  and backup_runs are left out of the backup itself.
- The panel polls only while a run is going, capped at the run limit plus
  five minutes.

The music switch named in the task is left to #6774, so no control ships
that does nothing.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-02 14:50:55 -05:00
bvandeusenandClaude Opus 5.5 faf19ef67b test: the file-name test's later backup is later in UTC too
release / govulncheck (push) Successful in 22s
release / web (push) Successful in 1m33s
release / go (push) Successful in 1m49s
release / android (push) Successful in 4m59s
release / integration (push) Successful in 5m27s
release / Build signed APK (push) Successful in 5m1s
release / Attach APK to the Release (tag releases only) (push) Skipped
release / Build + push container image (push) Successful in 14s
release / Verify release artifacts (tag releases only) (push) Skipped
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-02 14:01:31 -05:00
bvandeusenandClaude Opus 5.5 e11011e977 fix: the backup package checks or explicitly drops every error the linter flags
release / govulncheck (push) Successful in 24s
release / go (push) Failing after 1m42s
release / web (push) Successful in 1m25s
release / android (push) Successful in 4m52s
release / Build signed APK (push) Successful in 5m1s
release / integration (push) Canceled after 5m17s
release / Attach APK to the Release (tag releases only) (push) Canceled after 0s
release / Build + push container image (push) Canceled after 0s
release / Verify release artifacts (tag releases only) (push) Canceled after 0s
Closing a refused S3 response and a test file, and the fake bucket's
writes, now say they ignore the error on purpose; handler parameters the
tests don't use are named _.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-02 13:56:22 -05:00
bvandeusenandClaude Opus 5.5 b6ae205862 feat: Minstrel can put and list objects in a path-style S3 bucket, signed with SigV4 and no SDK (#6771)
release / govulncheck (push) Successful in 14s
release / android (push) Successful in 4m37s
release / Build signed APK (push) Successful in 4m47s
release / integration (push) Failing after 5m3s
release / Attach APK to the Release (tag releases only) (push) Skipped
release / Build + push container image (push) Skipped
release / Verify release artifacts (tag releases only) (push) Skipped
release / go (push) Failing after 1m7s
release / web (push) Successful in 1m14s
Bucket.Put stores a body under a key, signed over its SHA-256; Bucket.List
pages through ListObjectsV2. A backup only adds and lists, so an
append-only key is enough. Keys are encoded once, exactly as signed, so
music paths with spaces, plus signs and accents reach the bucket intact.
A refusal names S3's code in words ("the secret key is wrong").

The signer is pinned to AWS's published GET Object, PUT Object and
ListObjects examples.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-02 13:51:02 -05:00
bvandeusenandClaude Opus 5.5 8be69b752a feat: a backup is one uncompressed tar of per-table JSON Lines that stays the same bytes day to day (#6769)
The file is minstrel-<UTC stamp>.tar: manifest.json first, then
tables/<table>.jsonl, rows one per line in primary-key order. The run's
time lives only in the manifest and member headers are fixed, so a
deduplicating off-site copy stores unchanged tables once.

WriteFile writes to a hidden .part file, syncs it and renames it, so a
watched folder never shows half a backup. Read refuses anything that
isn't a whole backup: another format, a newer format version, a cut-off
file, or a member holding fewer rows than the manifest says.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-02 13:51:02 -05:00
bvandeusenandClaude Opus 5.5 a67897a6cd feat: Minstrel can read its irreplaceable data out as JSON Lines and load it back into an empty instance (#6770)
release / web (push) Successful in 1m17s
release / android (push) Successful in 4m41s
release / Build signed APK (push) Successful in 4m52s
release / govulncheck (push) Successful in 15s
release / go (push) Successful in 1m29s
release / integration (push) Successful in 5m3s
release / Attach APK to the Release (tag releases only) (push) Skipped
release / Build + push container image (push) Successful in 14s
release / Verify release artifacts (tag releases only) (push) Skipped
internal/backup names every table a backup holds, in parent-first order,
and every table it leaves out with the reason: what the server computes
from the music files, provider caches it fetches again, sign-ins and
one-time links, and logs of past runs. The library's identity rows travel
because every like, play and playlist entry hangs off a track id, and the
sync change feed travels because the apps' first sync reads it.

Sealed credentials are left out of the file, and so are the markers that
say tag and art enrichment are done, so a restored server asks for the
credentials again and redoes that work rather than believing it has it.

Snapshot reads one REPEATABLE READ, read-only transaction, each table in
primary-key order, keys sorted. Restore refuses an instance with accounts
or tracks and a backup from a newer schema, replaces seeded settings,
inserts by column intersection in batches, skips generated columns, and
carries serial sequences on past the restored ids.

Guards: every schema table is backed up or excluded; every backed-up
table comes after what it references and references nothing excluded;
every omitted column exists; a restore gives back the backup byte for byte.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-02 13:43:16 -05:00