DB maintenance + health observability + Postgres integration CI lane #72

Merged

bvandeusen merged 5 commits from dev into main 2026-06-14 20:54:34 -04:00

Conversation 0 Commits 5 Files Changed 12

+915 -4

5 Commits

Author	SHA1	Message	Date
bvandeusenandClaude Opus 4.8	4f31890bde	test(ci): dispose engine between integration tests (per-loop pool) ... First integration run proved the lane works (run_maintenance test passed against real Postgres), but the health test failed with 'Future attached to a different loop': pytest-asyncio uses a fresh loop per test while the app's module-level engine pools a connection from the prior test's loop. Dispose the engine in each test's teardown so the next test starts with an empty pool on its own loop. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-06-14 19:24:09 -04:00
bvandeusenandClaude Opus 4.8	2ad2e943f3	test(ci): add Postgres integration lane + real run_maintenance guard ... The unit suite can't catch sync/async API mismatches against SQLAlchemy (an un-awaited execution_options passed green CI but failed at runtime: VACUUM 0/6). Add a real-Postgres integration lane modelled on the family pattern (rules 6/79-82): a new CI 'integration' job with a postgres:16 service, bridge-IP discovery, busybox-safe readiness wait, and 'alembic upgrade head', running pytest -m integration. Non-gating, like the unit lane. - tests/test_integration_db_maintenance.py: runs run_maintenance() and get_table_health() against real Postgres; asserts all allowlisted tables vacuum OK (the await regression makes this fail) and health reports real stats. - pyproject: register the 'integration' marker. - conftest: integration-marked tests use the real DATABASE_URL, not the stub. - ci.yml: unit 'test' job now runs -m 'not integration'. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-06-14 19:19:18 -04:00
bvandeusenandClaude Opus 4.8	e6c89f6b88	fix(db): await AsyncConnection.execution_options in run_maintenance ... execution_options() is a coroutine on AsyncConnection and must be awaited; the un-awaited call returned a coroutine, so exec_driver_sql() blew up with AttributeError and every table's VACUUM was skipped (Run-now reported 0/6). A prior change had wrongly dropped the await. Fix it and make the test mock execution_options async so this call shape is actually exercised. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-06-14 18:31:30 -04:00
bvandeusenandClaude Opus 4.8	96079d5b77	feat(db): table-health readout — per-table bloat metrics in admin card ... You can't decide what to maintain without seeing what's bloating. Adds a read-only health panel driven by Postgres' own statistics views. - services/db_maintenance.py: get_table_health() queries pg_stat_user_tables + pg_total_relation_size + pg_database_size — per-table size, live/dead tuples, dead-tuple ratio (the bloat signal), and last (auto)vacuum/(auto)analyze. - routes/admin.py: admin-only GET /api/admin/db-maintenance/health. - SettingsView.vue: 'Table health' table in the maintenance card, all tables sorted by dead tuples, rows >=20% dead-ratio flagged; total DB size shown; refreshes after a Run-now so the dead-tuple drop is visible. - Tests: health row/size shaping + null-timestamp passthrough; route + service surface. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-06-14 17:53:24 -04:00
bvandeusenandClaude Opus 4.8	c4553d937c	feat(db): scheduled DB maintenance — daily targeted VACUUM (ANALYZE) ... Adds a daily off-hours VACUUM (ANALYZE) over the high-churn tables the retention/purge sweeps churn (app_logs, notifications, token tables, notes, note_versions), on top of Postgres autovacuum, to reclaim bloat left by the nightly bulk DELETEs and keep planner stats fresh. - services/db_maintenance.py: run_maintenance() over a closed table allowlist via an AUTOCOMMIT connection (VACUUM can't run in a txn); per-table summary persisted as the db_maintenance_last_run admin setting. - services/db_maintenance_scheduler.py: BackgroundScheduler cron (default 04:00 UTC, after the 03:30 trash purge); enabled-gate checked at fire time; live reschedule on hour change. Wired into app.py start/stop. - routes/admin.py: admin-only GET/PUT /api/admin/db-maintenance + POST /run. - settings.py: set_admin_setting() (write-side of get_admin_setting) for out-of-request writes. - SettingsView.vue: admin 'Database maintenance' card — enable toggle, run-hour (UTC), Run-now, last-run summary. - Tests: allowlist is closed, VACUUM issued per table, one failure doesn't abort the rest, summary persisted; route/scheduler/service surface. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-06-14 16:42:04 -04:00