feat(search): pgvector substrate — vector(384) + HNSW for semantic search

Move semantic_search_notes off the full-table Python cosine scan onto a native
pgvector column: indexed ORDER BY embedding <=> :q LIMIT k (HNSW, cosine).
Migration 0067 enables the extension, converts the JSONB embedding column to
vector(384) (stale-dim rows dropped and regenerated by the startup backfill),
and builds the HNSW cosine index. Postgres image moves postgres:16-alpine ->
pgvector/pgvector:pg17 across prod, quickstart, and CI.

Scribe: project 2, milestone 93, task 1031.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Xz4j1H7pjYSjKsEpgcNH5E
This commit is contained in:
2026-06-22 20:10:15 -04:00
parent 5fbee18a94
commit 513019786e
8 changed files with 217 additions and 26 deletions
+4 -2
View File
@@ -165,7 +165,9 @@ jobs:
SECRET_KEY: ci_integration_placeholder
services:
postgres:
image: postgres:16-alpine
# pgvector image so `alembic upgrade head` can run migration 0067
# (CREATE EXTENSION vector). PG17 — matches the prod/quickstart image.
image: pgvector/pgvector:pg17
env:
POSTGRES_USER: scribe
POSTGRES_PASSWORD: ci_integration
@@ -189,7 +191,7 @@ jobs:
set -eux
echo "=== container landscape (diagnostic for the name filter) ==="
docker ps -a --format '{{.ID}} {{.Image}} -> {{.Names}}'
PG=$(docker ps --filter "name=integration" --filter "ancestor=postgres:16-alpine" -q | head -n1)
PG=$(docker ps --filter "name=integration" --filter "ancestor=pgvector/pgvector:pg17" -q | head -n1)
test -n "$PG"
PG_IP=$(docker inspect -f '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' "$PG")
test -n "$PG_IP"