bvandeusenandClaude Opus 5.5
0cf8edaa10
CI and images / lint (push) Successful in 2s
CI and images / extension-version (push) Successful in 2s
CI and images / frontend-build (push) Successful in 25s
CI and images / extension-test (push) Successful in 18s
CI and images / backend-lint-and-test (push) Failing after 44s
CI and images / integration (push) Failing after 2m28s
CI and images / sign-extension (push) Skipped
CI and images / build-web (push) Skipped
CI and images / smoke-web (push) Skipped
CI and images / promote (push) Skipped
CI and images / refresh-dev (push) Skipped
CI and images / unattended-failure (push) Skipped
CI and images / build-agent (push) Skipped
With an off-site destination set up, the library's files are sent to the bucket's media/ folder as media/ab/cd/<sha256><ext>, beside the database dumps under database/. Each file goes once: a new backup_blob table records what each destination holds (migration 0121), and a run's work list is every image and attachment sha with no row yet, so runs are incremental and a new bucket starts its copy from the beginning. - The bucket checks each file against the sha recorded at import; a file over 1 GiB goes in parts and is checked before the upload is completed. A file missing or changed on disk is reported, not stored. - A run is chunked: each 45-minute chunk commits after every file and queues the next, so other long-maintenance jobs get turns. One run at a time. The stall sweep judges a media run by its progress heartbeat (backup_run.last_progress_at), not its start. - A file that can't be sent is listed on the run (status ok, with the error line) and tried again by the next run. A refused put for an object already in an append-only bucket counts as sent. - Nightly, with a destination set, runs the dump and then the media. - Weekly reconcile lists media/ and re-sends what the bucket lost, and adopts what it already holds (a restored database sends nothing twice). - Uploads use the bandwidth limit; read transactions end before transfers. - Backups card: with a destination, an "Images and attachments" section shows coverage (files and bytes in the bucket), "Send new files now", and the media runs. Without one, the local tarball section is unchanged. - Media run history keeps the latest 60 rows. Task #6879, milestone 586. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
75 lines
3.4 KiB
Python
75 lines
3.4 KiB
Python
"""FC-3h: backup_run — operator-facing artifact record for a backup run.
|
|
|
|
One row per backup attempt (kind='db', 'images', or 'media' for the off-site
|
|
upload of images and attachments, which spans many task chunks). Lifecycle
|
|
tracking (started_at/finished_at/duration_ms/exception text) lives
|
|
in task_run from FC-3i — this row records artifact metadata: file
|
|
paths, sizes, tag (retention protection), and restore lineage via
|
|
restored_from_id.
|
|
|
|
Status values (String, not Postgres ENUM — per
|
|
feedback_check_existing_enums):
|
|
pending — created but task hasn't started yet (rare; usually
|
|
status starts as 'running' from the task body).
|
|
running — backup task is in flight.
|
|
ok — artifact successfully written.
|
|
error — task raised; error column populated.
|
|
restoring — this row represents a restore attempt (kind = restored
|
|
kind); linked to source via restored_from_id.
|
|
restored — restore completed successfully.
|
|
"""
|
|
|
|
from datetime import datetime
|
|
|
|
from sqlalchemy import JSON, BigInteger, DateTime, ForeignKey, Index, Integer, String, Text, text
|
|
from sqlalchemy.orm import Mapped, mapped_column
|
|
|
|
from .base import Base
|
|
|
|
|
|
class BackupRun(Base):
|
|
__tablename__ = "backup_run"
|
|
|
|
|
|
__table_args__ = (
|
|
# alembic 0017: reporting indexes, never declared on the model (#3275).
|
|
Index("ix_backup_run_kind_started", "kind", text("started_at DESC")),
|
|
Index("ix_backup_run_status_finished", "status", text("finished_at DESC")),
|
|
Index("ix_backup_run_tag_partial", "tag", postgresql_where=text("tag IS NOT NULL")),
|
|
)
|
|
id: Mapped[int] = mapped_column(Integer, primary_key=True)
|
|
# No index=True: ix_backup_run_kind_started (above) already leads with
|
|
# `kind`, so a single-column index on it was pure write cost (#3301).
|
|
kind: Mapped[str] = mapped_column(String(16), nullable=False)
|
|
status: Mapped[str] = mapped_column(
|
|
# No index=True — ix_backup_run_status_finished leads with `status`.
|
|
String(16), nullable=False, default="pending",
|
|
server_default="pending",
|
|
)
|
|
tag: Mapped[str | None] = mapped_column(String(64), nullable=True, index=True)
|
|
triggered_by: Mapped[str] = mapped_column(String(32), nullable=False)
|
|
started_at: Mapped[datetime] = mapped_column(
|
|
DateTime(timezone=True), nullable=False, index=True,
|
|
)
|
|
finished_at: Mapped[datetime | None] = mapped_column(
|
|
DateTime(timezone=True), nullable=True, index=True,
|
|
)
|
|
sql_path: Mapped[str | None] = mapped_column(Text, nullable=True)
|
|
tar_path: Mapped[str | None] = mapped_column(Text, nullable=True)
|
|
size_bytes: Mapped[int | None] = mapped_column(BigInteger, nullable=True)
|
|
error: Mapped[str | None] = mapped_column(Text, nullable=True)
|
|
manifest: Mapped[dict] = mapped_column(
|
|
JSON, nullable=False, default=dict, server_default="{}",
|
|
)
|
|
# Self-referential FK, unindexed until 0089 (#3300): SET NULL has to find
|
|
# the rows pointing at a deleted run before it can null them.
|
|
# Heartbeat of a run that spans hours (the media backup): set as it makes
|
|
# progress, and what the stall sweep measures its silence from.
|
|
last_progress_at: Mapped[datetime | None] = mapped_column(
|
|
DateTime(timezone=True), nullable=True,
|
|
)
|
|
restored_from_id: Mapped[int | None] = mapped_column(
|
|
ForeignKey("backup_run.id", ondelete="SET NULL"),
|
|
nullable=True, index=True,
|
|
)
|