10 Commits

Include renames

Author	SHA1	Message	Date
bvandeusenandClaude Opus 5.5	598abeb8cf	fix: a fresh app install syncs the whole library, not the last 30 days of changes (#6778) ... The change feed dropped every row older than 30 days, and an app syncing from cursor 0 builds its library from the feed, so a fresh install, cleared data or a 410 reset got only what had changed lately. The compactor now keeps each entity's latest change however old. It drops a row once a newer one for the same entity exists, and a delete once it is 30 days old, raising a stored mark (library_changes_trimmed) past it; an app whose cursor is below the mark is sent back to 0, as before. Migration 0101 adds the index the compactor needs, seeds the mark where the old compactor left off, and gives every artist, album and track whose latest row is not an upsert a fresh one, so the rows the old compactor dropped come back. The backup carries the mark with the feed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-11-02 22:07:42 -05:00
bvandeusenandClaude Opus 5.5	68e1e47534	fix: a new server takes a backup after its first scan, and restarts to load the restored settings (#6786) ... A one-off end-to-end run of the :dev image (scratch Postgres, generated FLAC files, no outbound network) found that the web restore, done the way docs/hosting.md describes, was refused with not_empty: a new server's startup scan fills the library before anyone signs up, and Restore counted tracks as occupied. The command-line restore passed the same run. - Restore now asks only that the instance has no accounts. The library the first scan found is replaced by the backup's (every foreign key into artists/albums/tracks cascades or sets null), and the restore locks the library tables so a running scan waits rather than adding to them. - Settings are read once at boot by about ten services, the library folders among them, so after a web restore the server ran on its defaults until restarted. The restore endpoint now answers, then asks the process to restart in place (syscall.Exec of the same binary, arguments and environment), which needs no container restart policy. The startup scan of the restarted server replaces the scan the endpoint used to start. - The restore form says Minstrel is restarting; the docs say a scanned new server still takes a backup. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-11-02 20:21:00 -05:00
bvandeusenandClaude Opus 5.5	29042e4eb2	feat: a new Minstrel restores a backup, from the web app or the command line, then rescans (#6775) ... A new server (no accounts, no library) offers "Restore a backup" beside "Create account". The file goes up as the request body to POST /api/auth/restore with the setup token from the log in X-Setup-Token, since the backup's admins become this server's. It loads in one transaction, then a library scan starts and finds each file at its path. The same load runs offline as `minstrel restore FILE`, which migrates the schema first. That path alone gets a 1 GiB, 15-minute body allowance instead of 4 MiB and 30s, and skips the JSON-only check for cookie writes: a stale cookie from the old server would otherwise refuse the upload, and the custom header already rules out a cross-site form. The library sync answers 410 cursor_ahead to a cursor past the last one the server has handed out, read from the change sequence, so an app synced with the old server after the backup was made starts over instead of skipping the ids the restored server reuses. Restore now leaves each sequence as having issued the restored maximum, so that reading holds; a compacted log still counts its handed-out cursors. backup.Read keeps the underlying error in its chain, so an oversized upload answers 413 rather than "not a backup". The hosting docs say how to put the music back first and what is entered again afterwards. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-11-02 16:10:06 -05:00
bvandeusenandClaude Opus 5.5	967763f6ce	feat: the backup can include the music, sending each library file once to music/ and again only when it changes (#6774) ... With "Include the music" on, each night's run sends the library files the destination doesn't hold yet, after the database file has landed, for up to six hours; a large library spans several nights the first time. Files keep their path under the library folders' common root, beside database/ in the same folder or bucket. backup_music_files records each file's size and mtime as sent, so a night sends only new and changed files, and a rescan that touches a track but finds the same file sends nothing. A changed file goes to its old name; Minstrel never deletes there. An object the bucket refuses (a locked overwrite) is kept with its answer and goes again once the file changes or the settings are saved; three refusals in a row stop the night's music, since that is a key that may not write under music/. A folder copy is written under a .part name and renamed, keeping the file's time. The panel shows how much of the library is in, the refused files, and each run's music count and note. The verdict still turns on the database file. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-11-02 15:26:20 -05:00
bvandeusenandClaude Opus 5.5	9d324ab2d0	feat: Minstrel backs itself up nightly to a folder or an S3 bucket, off until an admin switches it on (#6773) ... Admin › Backups shows a verdict first (off, not set up, waiting, backed up, overdue, failing since when), Back up now, Test destination and the latest runs, with the settings beside them. - backup.Service: one run at a time, bounded at 30 minutes; the schedule ticks every 5 minutes, catches up a slot missed while the server was down, and retries a failure after an hour and then after as long as it has been failing. Runs are recorded in backup_runs, pruned after 90 days; a run cut off by a shutdown is marked failed at the next start. - A folder destination writes database/minstrel-<UTC stamp>.tar and refuses a folder that isn't there. An S3 destination stages the file, hashes it, and uploads it under database/ through a throttled reader (4 MiB/s by default, 0 for no limit). - Test destination checks the form as typed and only lists. - The bucket secret is sealed like every stored credential; backup_settings and backup_runs are left out of the backup itself. - The panel polls only while a run is going, capped at the run limit plus five minutes. The music switch named in the task is left to #6774, so no control ships that does nothing. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-11-02 14:50:55 -05:00
bvandeusenandClaude Opus 5.5	faf19ef67b	test: the file-name test's later backup is later in UTC too ... Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-11-02 14:01:31 -05:00
bvandeusenandClaude Opus 5.5	e11011e977	fix: the backup package checks or explicitly drops every error the linter flags ... Closing a refused S3 response and a test file, and the fake bucket's writes, now say they ignore the error on purpose; handler parameters the tests don't use are named _. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-11-02 13:56:22 -05:00
bvandeusenandClaude Opus 5.5	b6ae205862	feat: Minstrel can put and list objects in a path-style S3 bucket, signed with SigV4 and no SDK (#6771) ... Bucket.Put stores a body under a key, signed over its SHA-256; Bucket.List pages through ListObjectsV2. A backup only adds and lists, so an append-only key is enough. Keys are encoded once, exactly as signed, so music paths with spaces, plus signs and accents reach the bucket intact. A refusal names S3's code in words ("the secret key is wrong"). The signer is pinned to AWS's published GET Object, PUT Object and ListObjects examples. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-11-02 13:51:02 -05:00
bvandeusenandClaude Opus 5.5	8be69b752a	feat: a backup is one uncompressed tar of per-table JSON Lines that stays the same bytes day to day (#6769) ... The file is minstrel-<UTC stamp>.tar: manifest.json first, then tables/<table>.jsonl, rows one per line in primary-key order. The run's time lives only in the manifest and member headers are fixed, so a deduplicating off-site copy stores unchanged tables once. WriteFile writes to a hidden .part file, syncs it and renames it, so a watched folder never shows half a backup. Read refuses anything that isn't a whole backup: another format, a newer format version, a cut-off file, or a member holding fewer rows than the manifest says. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-11-02 13:51:02 -05:00
bvandeusenandClaude Opus 5.5	a67897a6cd	feat: Minstrel can read its irreplaceable data out as JSON Lines and load it back into an empty instance (#6770) ... internal/backup names every table a backup holds, in parent-first order, and every table it leaves out with the reason: what the server computes from the music files, provider caches it fetches again, sign-ins and one-time links, and logs of past runs. The library's identity rows travel because every like, play and playlist entry hangs off a track id, and the sync change feed travels because the apps' first sync reads it. Sealed credentials are left out of the file, and so are the markers that say tag and art enrichment are done, so a restored server asks for the credentials again and redoes that work rather than believing it has it. Snapshot reads one REPEATABLE READ, read-only transaction, each table in primary-key order, keys sorted. Restore refuses an instance with accounts or tracks and a backup from a newer schema, replaces seeded settings, inserts by column intersection in batches, skips generated columns, and carries serial sequences on past the restored ids. Guards: every schema table is backed up or excluded; every backed-up table comes after what it references and references nothing excluded; every omitted column exists; a restore gives back the backup byte for byte. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-11-02 13:43:16 -05:00