1e63f70bf71843dab0c40eb41a28e8da4960cba3
3
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
|
1e63f70bf7 |
fix: a media backup or restore cut off by a restart carries on where it stopped
CI and images / lint (push) Failing after 1s
CI and images / extension-version (push) Successful in 2s
CI and images / extension-test (push) Successful in 19s
CI and images / frontend-build (push) Successful in 20s
CI and images / backend-lint-and-test (push) Successful in 46s
CI and images / integration (push) Failing after 3m8s
CI and images / sign-extension (push) Skipped
CI and images / build-web (push) Skipped
CI and images / smoke-web (push) Skipped
CI and images / promote (push) Skipped
CI and images / refresh-dev (push) Skipped
CI and images / unattended-failure (push) Skipped
CI and images / build-agent (push) Skipped
A media backup and a restore's file phase run as chains of 45-minute chunks,
each queuing the next when it ends. A restart killed the chunk in flight, so
nothing queued the next one; the run's row stayed "running", "Back up now"
skipped it as already running, and only the 7-hour stall sweep ended it.
- When the long-maintenance lane starts, it queues the next chunk of every
media run still running and every restore fetching its files.
- Each chunk writes its token to the run and hands it to the chunk it queues.
A chunk queued by any other chunk is dropped (a killed chunk's message
redelivered by acks_late, say), so a resume never doubles the chain; a chunk
taken over stops at its next file.
- The integration job installs postgresql-client when the CI image lacks it:
the backup round trip in
|
||
|
|
d465198c4a |
fix: the media backup task has its own stall threshold, and its test seeds a post for a shared archive
CI and images / lint (push) Successful in 2s
CI and images / extension-version (push) Successful in 3s
CI and images / frontend-build (push) Successful in 27s
CI and images / extension-test (push) Successful in 23s
CI and images / backend-lint-and-test (push) Successful in 47s
CI and images / integration (push) Successful in 2m25s
CI and images / sign-extension (push) Successful in 2s
CI and images / build-agent (push) Successful in 5s
CI and images / build-web (push) Successful in 2m12s
CI and images / smoke-web (push) Successful in 50s
CI and images / refresh-dev (push) Skipped
CI and images / unattended-failure (push) Skipped
CI and images / promote (push) Successful in 2s
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
0cf8edaa10 |
feat: images and attachments go to the off-site bucket once each, by fingerprint
CI and images / lint (push) Successful in 2s
CI and images / frontend-build (push) Successful in 25s
CI and images / extension-test (push) Successful in 18s
CI and images / backend-lint-and-test (push) Failing after 44s
CI and images / extension-version (push) Successful in 2s
CI and images / integration (push) Failing after 2m28s
CI and images / sign-extension (push) Skipped
CI and images / build-web (push) Skipped
CI and images / smoke-web (push) Skipped
CI and images / promote (push) Skipped
CI and images / refresh-dev (push) Skipped
CI and images / unattended-failure (push) Skipped
CI and images / build-agent (push) Skipped
With an off-site destination set up, the library's files are sent to the bucket's media/ folder as media/ab/cd/<sha256><ext>, beside the database dumps under database/. Each file goes once: a new backup_blob table records what each destination holds (migration 0121), and a run's work list is every image and attachment sha with no row yet, so runs are incremental and a new bucket starts its copy from the beginning. - The bucket checks each file against the sha recorded at import; a file over 1 GiB goes in parts and is checked before the upload is completed. A file missing or changed on disk is reported, not stored. - A run is chunked: each 45-minute chunk commits after every file and queues the next, so other long-maintenance jobs get turns. One run at a time. The stall sweep judges a media run by its progress heartbeat (backup_run.last_progress_at), not its start. - A file that can't be sent is listed on the run (status ok, with the error line) and tried again by the next run. A refused put for an object already in an append-only bucket counts as sent. - Nightly, with a destination set, runs the dump and then the media. - Weekly reconcile lists media/ and re-sends what the bucket lost, and adopts what it already holds (a restored database sends nothing twice). - Uploads use the bandwidth limit; read transactions end before transfers. - Backups card: with a destination, an "Images and attachments" section shows coverage (files and bytes in the bucket), "Send new files now", and the media runs. Without one, the local tarball section is unchanged. - Media run history keeps the latest 60 rows. Task #6879, milestone 586. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |