2 Commits

Include renames

Author	SHA1	Message	Date
bvandeusenandClaude Fable 5	0e70a3896b	feat(embeddings): per-chunk rows — schema, write path, version-aware backfill (#280 steps 2+3) ... note_embeddings becomes one row per chunk: PK (note_id, chunk_index), plus chunk_text (what this vector actually encodes) and chunker_version. Migration 0077 clears the table — embeddings are derived (0067 precedent) and the old whole-document rows are indistinguishable from single-chunk notes, so the startup backfill regenerates the corpus at the new shape. The backfill is now version-aware: a future shape change is a CHUNKER_VERSION bump that re-embeds exactly the stale notes, not another wipe. upsert_note_embedding takes (title, body) and chunks internally — one path for the write path, the recurrence spawn and the backfill. The recurrence spawn's own embed call is deleted outright: create_note already embeds via embed_note (#2056), so the spawn was a second copy of the rule. An emptied record now CLEARS its stale vectors instead of leaving them findable. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UaYUaouG9jjhATyuxCKrQs	2026-08-08 23:47:34 -04:00
bvandeusenandClaude Fable 5	6b5043a69c	feat(embeddings): chunk_document — the chunked document shape (#280 step 1) ... bge-small reads 512 tokens and fastembed truncates silently, so a single whole-document vector permanently lost everything past ~400 words. The new shape: split at markdown headings (fence-aware), merge small sections, split oversize ones at paragraph boundaries, title-anchor every chunk, repeat the section heading on continuation pieces. A record that fits the window yields exactly one chunk identical to the historical title\nbody shape, so the corpus's sharpest records are byte-for-byte unaffected. CHUNKER_VERSION added so later shape changes re-embed by version comparison instead of a table wipe. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UaYUaouG9jjhATyuxCKrQs	2026-08-08 23:43:51 -04:00