feat(db): scheduled DB maintenance — daily targeted VACUUM (ANALYZE)
CI & Build / Python lint (push) Successful in 3s
CI & Build / TypeScript typecheck (push) Successful in 33s
CI & Build / Python tests (push) Successful in 48s
CI & Build / Build & push image (push) Successful in 54s

Adds a daily off-hours VACUUM (ANALYZE) over the high-churn tables the
retention/purge sweeps churn (app_logs, notifications, token tables, notes,
note_versions), on top of Postgres autovacuum, to reclaim bloat left by the
nightly bulk DELETEs and keep planner stats fresh.

- services/db_maintenance.py: run_maintenance() over a closed table allowlist
  via an AUTOCOMMIT connection (VACUUM can't run in a txn); per-table summary
  persisted as the db_maintenance_last_run admin setting.
- services/db_maintenance_scheduler.py: BackgroundScheduler cron (default
  04:00 UTC, after the 03:30 trash purge); enabled-gate checked at fire time;
  live reschedule on hour change. Wired into app.py start/stop.
- routes/admin.py: admin-only GET/PUT /api/admin/db-maintenance + POST /run.
- settings.py: set_admin_setting() (write-side of get_admin_setting) for
  out-of-request writes.
- SettingsView.vue: admin 'Database maintenance' card — enable toggle, run-hour
  (UTC), Run-now, last-run summary.
- Tests: allowlist is closed, VACUUM issued per table, one failure doesn't
  abort the rest, summary persisted; route/scheduler/service surface.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
2026-06-14 16:42:04 -04:00
parent ee02ed37c1
commit c4553d937c
8 changed files with 572 additions and 1 deletions
+115
View File
@@ -0,0 +1,115 @@
"""Basic Postgres maintenance — targeted VACUUM (ANALYZE).
Postgres autovacuum already reclaims dead tuples and refreshes planner stats
in the background. This adds a daily, off-hours top-up over the handful of
tables the retention/purge sweeps churn hardest (log retention, notification
purge, auth-token purge, trash purge, version pruning) — so the bloat those
bulk DELETEs leave behind is collected promptly and the planner keeps good
stats. It is deliberately narrow; the rest of the schema is left to autovacuum.
Security: table names cannot be parameterized in SQL, so VACUUM is only ever
issued against names from the hardcoded MAINTENANCE_TABLES allowlist — there is
no path from user input to the table name.
"""
from __future__ import annotations
import json
import logging
import time
from datetime import datetime, timezone
from scribe.models import engine
from scribe.services.settings import get_admin_setting, set_admin_setting
logger = logging.getLogger(__name__)
# High-churn tables — exactly the ones the scheduled delete sweeps hit:
# app_logs ← log retention (hourly)
# notifications ← read-notification purge (hourly)
# password_reset_tokens ← auth-token purge (daily)
# invitation_tokens ← auth-token purge (daily)
# notes ← trash purge (daily) + general churn
# note_versions ← version-pinning prune (daily)
MAINTENANCE_TABLES: tuple[str, ...] = (
"app_logs",
"notifications",
"password_reset_tokens",
"invitation_tokens",
"notes",
"note_versions",
)
_LAST_RUN_KEY = "db_maintenance_last_run"
async def run_maintenance(tables: tuple[str, ...] | list[str] | None = None) -> dict:
"""Run VACUUM (ANALYZE) over the allowlisted high-churn tables.
VACUUM cannot run inside a transaction block, so this uses a dedicated
AUTOCOMMIT connection rather than the usual session. Each table is
vacuumed independently; one failure is logged and does not abort the rest.
Returns a summary dict (also persisted as the db_maintenance_last_run
admin setting) of shape:
{"started_at": iso, "elapsed_ms": int, "tables": [
{"table": str, "ok": bool, "elapsed_ms": int, "error": str|None}
]}
"""
# Only ever operate on the closed allowlist — never an arbitrary name.
requested = tuple(tables) if tables is not None else MAINTENANCE_TABLES
targets = [t for t in requested if t in MAINTENANCE_TABLES]
started = time.monotonic()
started_at = datetime.now(timezone.utc).isoformat()
results: list[dict] = []
async with engine.connect() as conn:
# execution_options() is synchronous on AsyncConnection; it returns the
# connection with AUTOCOMMIT set (VACUUM can't run in a transaction).
autocommit = conn.execution_options(isolation_level="AUTOCOMMIT")
for table in targets:
t0 = time.monotonic()
try:
# table is from the allowlist above — safe to interpolate.
await autocommit.exec_driver_sql(f"VACUUM (ANALYZE) {table}")
results.append({
"table": table,
"ok": True,
"elapsed_ms": int((time.monotonic() - t0) * 1000),
"error": None,
})
except Exception as exc: # noqa: BLE001 — record per-table, keep going
logger.exception("VACUUM (ANALYZE) failed for %s", table)
results.append({
"table": table,
"ok": False,
"elapsed_ms": int((time.monotonic() - t0) * 1000),
"error": str(exc),
})
summary = {
"started_at": started_at,
"elapsed_ms": int((time.monotonic() - started) * 1000),
"tables": results,
}
ok = sum(1 for r in results if r["ok"])
logger.info(
"DB maintenance: vacuumed %d/%d table(s) in %dms",
ok, len(results), summary["elapsed_ms"],
)
try:
await set_admin_setting(_LAST_RUN_KEY, json.dumps(summary))
except Exception: # noqa: BLE001 — persisting the summary is best-effort
logger.warning("Failed to persist db_maintenance last-run summary", exc_info=True)
return summary
async def get_last_run() -> dict | None:
"""Return the most recent run summary, or None if it has never run."""
raw = await get_admin_setting(_LAST_RUN_KEY, "")
if not raw:
return None
try:
return json.loads(raw)
except (ValueError, TypeError):
return None