feat(embeddings): per-chunk rows — schema, write path, version-aware backfill (#280 steps 2+3)

note_embeddings becomes one row per chunk: PK (note_id, chunk_index), plus
chunk_text (what this vector actually encodes) and chunker_version. Migration
0077 clears the table — embeddings are derived (0067 precedent) and the old
whole-document rows are indistinguishable from single-chunk notes, so the
startup backfill regenerates the corpus at the new shape. The backfill is now
version-aware: a future shape change is a CHUNKER_VERSION bump that re-embeds
exactly the stale notes, not another wipe.

upsert_note_embedding takes (title, body) and chunks internally — one path
for the write path, the recurrence spawn and the backfill. The recurrence
spawn's own embed call is deleted outright: create_note already embeds via
embed_note (#2056), so the spawn was a second copy of the rule. An emptied
record now CLEARS its stale vectors instead of leaving them findable.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UaYUaouG9jjhATyuxCKrQs
This commit is contained in:
2026-08-08 23:47:34 -04:00
co-authored by Claude Fable 5
parent 6b5043a69c
commit 0e70a3896b
9 changed files with 277 additions and 41 deletions
+6 -5
View File
@@ -33,11 +33,12 @@ def embed_note(note) -> None:
try:
import asyncio
from scribe.services.embeddings import embedding_text, upsert_note_embedding
text = embedding_text(note.title, note.body)
if not text:
return
asyncio.create_task(upsert_note_embedding(note.id, note.user_id, text))
from scribe.services.embeddings import upsert_note_embedding
# Chunking and the empty-record gate live inside upsert_note_embedding —
# one path for every writer (#280).
asyncio.create_task(
upsert_note_embedding(note.id, note.user_id, note.title, note.body)
)
except RuntimeError:
pass # no running loop — a sync caller, not a failure
except Exception: # noqa: BLE001 - never let indexing break a write