fix(backup): compressed -Fc dumps + pg_restore; reconcile subprocess timeouts (#739)
CI / lint (push) Successful in 3s
CI / frontend-build (push) Successful in 20s
CI / backend-lint-and-test (push) Successful in 34s
CI / integration (push) Successful in 3m14s

DB backup polish (plan-task #764 Q3):
- pg_dump now uses custom format (-Fc): compressed (much smaller on NFS) and
  restored via pg_restore. Artifact extension .sql → .dump; restore_db swaps
  psql -f for pg_restore -d. BackupRun.sql_path field name kept (it's just the
  db artifact path).
- Reconcile the subprocess guardrails: the DB timeout was 720s with a stale
  'Celery soft is 10 min' comment, but backup_db_task's soft limit is actually
  1800s — so the bounded-kill fired 18 min early. Set DB=1700s / images=21000s,
  each just under its task's Celery soft limit so _run_bounded stays the
  primary guard (an NFS D-state hang defeats Celery's own SIGKILL).

Real shrink of the DB is the #764 prune; this makes each dump smaller/faster
on top of that.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-06-10 14:29:17 -04:00
parent d55e52ae9b
commit 7a40a50fe9
3 changed files with 37 additions and 19 deletions
+15 -7
View File
@@ -69,7 +69,7 @@ def test_backup_db_writes_sql_and_manifest(tmp_path, fake_subprocess):
)
sql = Path(result["sql_path"])
manifest = Path(result["manifest_path"])
assert sql.is_file() and sql.suffix == ".sql"
assert sql.is_file() and sql.suffix == ".dump"
assert manifest.is_file() and manifest.suffix == ".json"
assert result["kind"] == "db"
assert result["tar_path"] is None
@@ -92,6 +92,12 @@ def test_backup_db_strips_sqlalchemy_psycopg_driver(tmp_path, fake_subprocess):
assert "+psycopg" not in cmd[-1]
def test_backup_db_uses_compressed_custom_format(tmp_path, fake_subprocess):
# -Fc → pg_restore-loadable, compressed dump (#739 backup polish).
backup_service.backup_db(db_url="postgresql://u@h/d", images_root=tmp_path)
assert "-Fc" in fake_subprocess[0]
def test_backup_db_strips_asyncpg_driver(tmp_path, fake_subprocess):
backup_service.backup_db(
db_url="postgresql+asyncpg://u:p@h/d", images_root=tmp_path,
@@ -132,17 +138,19 @@ def test_backup_images_excludes_backups_and_quarantine(tmp_path, fake_subprocess
def test_restore_db_drops_schema_then_loads(tmp_path, fake_subprocess):
sql_path = tmp_path / "fake.sql"
sql_path.write_text("SELECT 1;")
dump_path = tmp_path / "fake.dump"
dump_path.write_bytes(b"\x00fake custom dump")
backup_service.restore_db(
db_url="postgresql://u@h/d", sql_path=sql_path,
db_url="postgresql://u@h/d", sql_path=dump_path,
)
# Two psql calls: one with -c (DROP SCHEMA), one with -f (load).
# Two calls: psql -c (DROP SCHEMA), then pg_restore -d (load the dump).
assert len(fake_subprocess) == 2
assert fake_subprocess[0][0] == "psql"
assert "-c" in fake_subprocess[0]
assert "DROP SCHEMA IF EXISTS public CASCADE" in fake_subprocess[0][-1]
assert "-f" in fake_subprocess[1]
assert str(sql_path) in fake_subprocess[1]
assert fake_subprocess[1][0] == "pg_restore"
assert "-d" in fake_subprocess[1]
assert str(dump_path) in fake_subprocess[1]
# --- restore_images --------------------------------------------------