feat: Minstrel backs itself up nightly to a folder or an S3 bucket, off until an admin switches it on (#6773)
release / web (push) Failing after 15s
release / govulncheck (push) Successful in 16s
release / go (push) Failing after 41s
release / integration (push) Failing after 3m29s
release / android (push) Successful in 4m7s
release / Attach APK to the Release (tag releases only) (push) Canceled after 0s
release / Build + push container image (push) Canceled after 0s
release / Verify release artifacts (tag releases only) (push) Canceled after 0s
release / Build signed APK (push) Canceled after 5m34s
release / web (push) Failing after 15s
release / govulncheck (push) Successful in 16s
release / go (push) Failing after 41s
release / integration (push) Failing after 3m29s
release / android (push) Successful in 4m7s
release / Attach APK to the Release (tag releases only) (push) Canceled after 0s
release / Build + push container image (push) Canceled after 0s
release / Verify release artifacts (tag releases only) (push) Canceled after 0s
release / Build signed APK (push) Canceled after 5m34s
Admin › Backups shows a verdict first (off, not set up, waiting, backed up, overdue, failing since when), Back up now, Test destination and the latest runs, with the settings beside them. - backup.Service: one run at a time, bounded at 30 minutes; the schedule ticks every 5 minutes, catches up a slot missed while the server was down, and retries a failure after an hour and then after as long as it has been failing. Runs are recorded in backup_runs, pruned after 90 days; a run cut off by a shutdown is marked failed at the next start. - A folder destination writes database/minstrel-<UTC stamp>.tar and refuses a folder that isn't there. An S3 destination stages the file, hashes it, and uploads it under database/ through a throttled reader (4 MiB/s by default, 0 for no limit). - Test destination checks the form as typed and only lists. - The bucket secret is sealed like every stored credential; backup_settings and backup_runs are left out of the backup itself. - The panel polls only while a run is going, capped at the run limit plus five minutes. The music switch named in the task is left to #6774, so no control ships that does nothing. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,50 @@
|
||||
package backup
|
||||
|
||||
import (
|
||||
"context"
|
||||
"io"
|
||||
"time"
|
||||
)
|
||||
|
||||
// Throttle returns a reader that reads r no faster than perSecond bytes a
|
||||
// second, so an upload leaves room on the connection for everything else.
|
||||
// perSecond of 0 or less returns r unchanged. A wait ends early when ctx does,
|
||||
// and the read then returns ctx's error.
|
||||
func Throttle(ctx context.Context, r io.Reader, perSecond int64) io.Reader {
|
||||
if perSecond <= 0 {
|
||||
return r
|
||||
}
|
||||
return &throttled{ctx: ctx, r: r, perSecond: perSecond}
|
||||
}
|
||||
|
||||
type throttled struct {
|
||||
ctx context.Context
|
||||
r io.Reader
|
||||
perSecond int64
|
||||
start time.Time
|
||||
sent int64
|
||||
}
|
||||
|
||||
func (t *throttled) Read(p []byte) (int, error) {
|
||||
if t.start.IsZero() {
|
||||
t.start = time.Now()
|
||||
}
|
||||
// Reads are cut to a tenth of a second's worth, so the pace is even
|
||||
// rather than a burst followed by a long pause.
|
||||
if chunk := max(t.perSecond/10, 1); int64(len(p)) > chunk {
|
||||
p = p[:chunk]
|
||||
}
|
||||
n, err := t.r.Read(p)
|
||||
t.sent += int64(n)
|
||||
at := t.start.Add(time.Duration(float64(t.sent) / float64(t.perSecond) * float64(time.Second)))
|
||||
if wait := time.Until(at); wait > 0 {
|
||||
timer := time.NewTimer(wait)
|
||||
defer timer.Stop()
|
||||
select {
|
||||
case <-t.ctx.Done():
|
||||
return n, t.ctx.Err()
|
||||
case <-timer.C:
|
||||
}
|
||||
}
|
||||
return n, err
|
||||
}
|
||||
Reference in New Issue
Block a user