Commit Graph
2 Commits
Author SHA1 Message Date
bvandeusenandClaude Fable 5 0e70a3896b feat(embeddings): per-chunk rows — schema, write path, version-aware backfill (#280 steps 2+3)
note_embeddings becomes one row per chunk: PK (note_id, chunk_index), plus
chunk_text (what this vector actually encodes) and chunker_version. Migration
0077 clears the table — embeddings are derived (0067 precedent) and the old
whole-document rows are indistinguishable from single-chunk notes, so the
startup backfill regenerates the corpus at the new shape. The backfill is now
version-aware: a future shape change is a CHUNKER_VERSION bump that re-embeds
exactly the stale notes, not another wipe.

upsert_note_embedding takes (title, body) and chunks internally — one path
for the write path, the recurrence spawn and the backfill. The recurrence
spawn's own embed call is deleted outright: create_note already embeds via
embed_note (#2056), so the spawn was a second copy of the rule. An emptied
record now CLEARS its stale vectors instead of leaving them findable.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UaYUaouG9jjhATyuxCKrQs
2026-08-08 23:47:34 -04:00
bvandeusenandClaude Opus 5 5c51e29f26 fix(embeddings): embed in the service, so every caller gets it (#2056)
CI & Build / Python lint (push) Successful in 3s
CI & Build / Python tests (push) Successful in 57s
CI & Build / Plugin hooks (push) Successful in 8s
CI & Build / integration (push) Successful in 15s
CI & Build / TypeScript typecheck (push) Successful in 33s
CI & Build / Build & push image (push) Successful in 29s
A note or task created through MCP was not semantically searchable until the
next restart's backfill ran. Embedding fired at the five REST route handlers
and nowhere else; the MCP tools call the service directly, so they skipped it.

The shape of this bug is the reason to care: it is invisible on an instance
that redeploys constantly (this one does, per rule 46) and permanent on one
that doesn't. Rule 115 — the product has to stand up for the install that
restarts twice a year, not just for the one that restarts hourly.

Moved to services/notes.embed_note(), called from create_note and update_note,
and deleted from all five routes. Every caller — REST, MCP, recurrence,
snippets — now gets it by construction rather than by remembering.

Two things fall out of having one implementation instead of six:

- It uses note.user_id, the OWNER. The routes were inconsistent: some passed
  the caller's uid, some the owner's. On a shared record the caller's id mints
  a second embedding row that nothing reads.
- services/snippets.py's _embed_snippet existed only because snippets are
  created via MCP and the routes couldn't cover them. Every one of its four
  call sites goes through notes_svc, so the helper and its four calls are gone,
  along with the eight test patches that existed to neutralise it.

RuntimeError (no running loop — unit tests, scripts) and any indexing failure
are both swallowed: a write that succeeded must not be failed by its index.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UaYUaouG9jjhATyuxCKrQs
2026-07-31 22:46:57 -04:00