perf(search): offload cosine scoring off event loop; document best-effort feed
CI & Build / Python lint (push) Successful in 3s
CI & Build / TypeScript typecheck (push) Successful in 38s
CI & Build / Build & push image (push) Has been cancelled
CI & Build / Python tests (push) Has been cancelled

Drift-audit Group 9 (param-cliff / unbounded search work):

- semantic_search_notes: the O(rows) cosine-similarity scoring loop ran
  synchronously on the event loop, so every RAG injection / search stalled
  other requests proportional to the user's embedding count. Move the scoring
  into asyncio.to_thread (results unchanged). The deeper fix — bounding the
  candidate set via pgvector ORDER BY/LIMIT — is noted as separate infra work.
- _semantic_knowledge_search: documented the best-effort top-N semantics —
   is the capped candidate-window size (not the true match count),
  matches beyond the cap aren't page-reachable, and each page recomputes the
  full merge. Prevents the silent-truncation trap; cached ranked-id paging /
  pgvector is the fix if exhaustive pagination is ever required.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-06-02 19:24:57 -04:00
parent 7ce5bb8450
commit 8d739c5da1
2 changed files with 23 additions and 10 deletions
+15 -10
View File
@@ -153,17 +153,22 @@ async def semantic_search_notes(
if not rows:
return []
scored: list[tuple[float, Note]] = []
for ne, note in rows:
try:
sim = _cosine_similarity(query_vec, ne.embedding)
except Exception:
continue
if sim >= threshold:
scored.append((sim, note))
def _score() -> list[tuple[float, Note]]:
out: list[tuple[float, Note]] = []
for ne, note in rows:
try:
sim = _cosine_similarity(query_vec, ne.embedding)
except Exception:
continue
if sim >= threshold:
out.append((sim, note))
out.sort(key=lambda x: x[0], reverse=True)
return out[:limit]
scored.sort(key=lambda x: x[0], reverse=True)
return scored[:limit]
# Offload the O(rows) cosine scoring off the event loop so a large corpus
# doesn't stall other requests while ranking. Results are unchanged; the
# real scaling fix (ORDER BY / LIMIT in pgvector) is a separate effort.
return await asyncio.to_thread(_score)
async def backfill_note_embeddings() -> None:
@@ -140,6 +140,14 @@ async def _semantic_knowledge_search(
Exact keyword matches always rank above semantic-only matches so that
searching for a name like "Weston" surfaces the note with that title
before conceptually related notes.
BEST-EFFORT TOP-N, not exhaustive pagination: the ranked candidate set is
capped (keyword limit*2 + up to ~200 semantic), so `total` is the size of
that window, NOT the true match count, and matches beyond the cap are not
reachable by paging. Each page also recomputes the full merge (O(corpus)
per page). Acceptable for an interactive "best results" feed; a cached
ranked-id list or pgvector ORDER BY/LIMIT is the fix if exhaustive,
cheap pagination is ever needed.
"""
# 1. Keyword search — title and body ILIKE
keyword_notes: list[Note] = []