Chunked embeddings — no record content invisible to search (#280) #104

Merged
bvandeusen merged 4 commits from dev into main 2026-08-08 23:54:34 -04:00
Owner

The embedding model reads at most 512 tokens and fastembed truncates the rest silently, so the one-vector-per-note shape permanently lost everything past ~400 words of a record — for a long dev-log, three quarters of its content could never influence search. Operator's call: no authoring bandaid, build the infrastructure right.

The shapechunk_document(title, body): fence-aware markdown-heading sections, small sections merged, oversize sections split at paragraph boundaries, title anchoring every chunk, section headings repeated on continuation pieces. A record that fits the window yields exactly one chunk identical to the historical shape — snippets and reference notes are byte-for-byte unaffected.

Storage — migration 0077: note_embeddings PK becomes (note_id, chunk_index) + chunk_text + chunker_version; table cleared (derived data, 0067 precedent). The startup backfill is version-aware: future shape changes are a CHUNKER_VERSION bump that re-embeds exactly the stale notes, not another wipe.

Retrieval — best chunk wins, everywhere: semantic_search_notes over-fetches chunk rows on the HNSW index (×4, composing with the ×3 supersession over-fetch) and collapses to best-chunk-per-note, inherited by every ranked consumer; list-q swaps its join for a correlated MIN-distance subquery; the duplicate report groups its self-join by note pair on MIN(distance); the write gate queries per candidate chunk (capped at 8), so a note duplicating an existing record in one section is now caught.

Deploy notes: migration 0077 runs on boot; first boot then re-embeds the whole corpus in the background (in-process model, minutes). Semantic search degrades gracefully to keyword during the backfill window.

CI green on head 23cb396 (run 3553), including the integration lane running 0077 + a two-chunk collapse test against real pgvector.

🤖 Generated with Claude Code

https://claude.ai/code/session_01UaYUaouG9jjhATyuxCKrQs

The embedding model reads at most 512 tokens and fastembed truncates the rest silently, so the one-vector-per-note shape permanently lost everything past ~400 words of a record — for a long dev-log, three quarters of its content could never influence search. Operator's call: no authoring bandaid, build the infrastructure right. **The shape** — `chunk_document(title, body)`: fence-aware markdown-heading sections, small sections merged, oversize sections split at paragraph boundaries, title anchoring every chunk, section headings repeated on continuation pieces. A record that fits the window yields exactly one chunk identical to the historical shape — snippets and reference notes are byte-for-byte unaffected. **Storage** — migration **0077**: `note_embeddings` PK becomes `(note_id, chunk_index)` + `chunk_text` + `chunker_version`; table cleared (derived data, 0067 precedent). The startup backfill is version-aware: future shape changes are a `CHUNKER_VERSION` bump that re-embeds exactly the stale notes, not another wipe. **Retrieval** — best chunk wins, everywhere: `semantic_search_notes` over-fetches chunk rows on the HNSW index (×4, composing with the ×3 supersession over-fetch) and collapses to best-chunk-per-note, inherited by every ranked consumer; list-q swaps its join for a correlated MIN-distance subquery; the duplicate report groups its self-join by note pair on MIN(distance); the write gate queries per candidate chunk (capped at 8), so a note duplicating an existing record in one section is now caught. **Deploy notes**: migration 0077 runs on boot; first boot then re-embeds the whole corpus in the background (in-process model, minutes). Semantic search degrades gracefully to keyword during the backfill window. CI green on head `23cb396` (run 3553), including the integration lane running 0077 + a two-chunk collapse test against real pgvector. 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_01UaYUaouG9jjhATyuxCKrQs
bvandeusen added 4 commits 2026-08-08 23:54:29 -04:00
feat(embeddings): chunk_document — the chunked document shape (#280 step 1)
CI & Build / Plugin hooks (push) Successful in 7s
CI & Build / Python tests (push) Successful in 46s
CI & Build / Python lint (push) Successful in 4s
CI & Build / integration (push) Successful in 16s
CI & Build / TypeScript typecheck (push) Successful in 32s
CI & Build / Build & push image (push) Successful in 25s
6b5043a69c
bge-small reads 512 tokens and fastembed truncates silently, so a single
whole-document vector permanently lost everything past ~400 words. The new
shape: split at markdown headings (fence-aware), merge small sections, split
oversize ones at paragraph boundaries, title-anchor every chunk, repeat the
section heading on continuation pieces. A record that fits the window yields
exactly one chunk identical to the historical title\nbody shape, so the
corpus's sharpest records are byte-for-byte unaffected. CHUNKER_VERSION added
so later shape changes re-embed by version comparison instead of a table wipe.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UaYUaouG9jjhATyuxCKrQs
note_embeddings becomes one row per chunk: PK (note_id, chunk_index), plus
chunk_text (what this vector actually encodes) and chunker_version. Migration
0077 clears the table — embeddings are derived (0067 precedent) and the old
whole-document rows are indistinguishable from single-chunk notes, so the
startup backfill regenerates the corpus at the new shape. The backfill is now
version-aware: a future shape change is a CHUNKER_VERSION bump that re-embeds
exactly the stale notes, not another wipe.

upsert_note_embedding takes (title, body) and chunks internally — one path
for the write path, the recurrence spawn and the backfill. The recurrence
spawn's own embed call is deleted outright: create_note already embeds via
embed_note (#2056), so the spawn was a second copy of the rule. An emptied
record now CLEARS its stale vectors instead of leaving them findable.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UaYUaouG9jjhATyuxCKrQs
feat(embeddings): best-chunk-per-note on every retrieval surface (#280 step 4)
CI & Build / Plugin hooks (push) Failing after 1s
CI & Build / Python lint (push) Failing after 3s
CI & Build / integration (push) Successful in 17s
CI & Build / TypeScript typecheck (push) Successful in 32s
CI & Build / Python tests (push) Successful in 46s
CI & Build / Build & push image (push) Skipped
041d8defbc
A note's relevance is now its best chunk's similarity, everywhere:

- semantic_search_notes keeps the indexed raw-distance top-k and over-fetches
  chunk rows (x4, composing with the x3 supersession over-fetch), then
  collapses to first-appearance-per-note — rows arrive distance-ordered, so
  first is best. Every ranked consumer (MCP/REST search, Browse, auto-inject,
  write-path, gate) inherits through the one function.
- list_notes semantic q swaps its join for a correlated MIN-distance
  subquery — the join would have repeated a long note once per matching chunk
  and made total count chunks.
- the duplicate report groups its self-join by note pair on MIN(distance):
  pair similarity = closest chunk pair, and the < join now also drops
  cross-chunk self-pairs that would flag every long note against itself.
- the write gate queries once per chunk of the candidate (capped at 8), so a
  note duplicating an existing record in ONE SECTION is caught — the
  whole-document query diluted exactly the section that mattered.

Integration test now seeds a two-chunk note and pins the collapse against
real pgvector.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UaYUaouG9jjhATyuxCKrQs
fix(recurrence): drop the now-unused child binding
CI & Build / Python lint (push) Successful in 2s
CI & Build / Plugin hooks (push) Successful in 8s
CI & Build / integration (push) Successful in 12s
CI & Build / TypeScript typecheck (push) Successful in 23s
CI & Build / Python tests (push) Successful in 46s
CI & Build / Build & push image (push) Successful in 27s
23cb396e65
The spawn's explicit embed call went with #280 step 3; the create_note result
had no other reader.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UaYUaouG9jjhATyuxCKrQs
bvandeusen merged commit 063494094a into main 2026-08-08 23:54:34 -04:00
Sign in to join this conversation.
No Reviewers
No labels
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: bvandeusen/FabledScribe#104