perf(search): offload cosine scoring off event loop; document best-effort feed
Drift-audit Group 9 (param-cliff / unbounded search work): - semantic_search_notes: the O(rows) cosine-similarity scoring loop ran synchronously on the event loop, so every RAG injection / search stalled other requests proportional to the user's embedding count. Move the scoring into asyncio.to_thread (results unchanged). The deeper fix — bounding the candidate set via pgvector ORDER BY/LIMIT — is noted as separate infra work. - _semantic_knowledge_search: documented the best-effort top-N semantics — is the capped candidate-window size (not the true match count), matches beyond the cap aren't page-reachable, and each page recomputes the full merge. Prevents the silent-truncation trap; cached ranked-id paging / pgvector is the fix if exhaustive pagination is ever required. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -153,17 +153,22 @@ async def semantic_search_notes(
|
||||
if not rows:
|
||||
return []
|
||||
|
||||
scored: list[tuple[float, Note]] = []
|
||||
for ne, note in rows:
|
||||
try:
|
||||
sim = _cosine_similarity(query_vec, ne.embedding)
|
||||
except Exception:
|
||||
continue
|
||||
if sim >= threshold:
|
||||
scored.append((sim, note))
|
||||
def _score() -> list[tuple[float, Note]]:
|
||||
out: list[tuple[float, Note]] = []
|
||||
for ne, note in rows:
|
||||
try:
|
||||
sim = _cosine_similarity(query_vec, ne.embedding)
|
||||
except Exception:
|
||||
continue
|
||||
if sim >= threshold:
|
||||
out.append((sim, note))
|
||||
out.sort(key=lambda x: x[0], reverse=True)
|
||||
return out[:limit]
|
||||
|
||||
scored.sort(key=lambda x: x[0], reverse=True)
|
||||
return scored[:limit]
|
||||
# Offload the O(rows) cosine scoring off the event loop so a large corpus
|
||||
# doesn't stall other requests while ranking. Results are unchanged; the
|
||||
# real scaling fix (ORDER BY / LIMIT in pgvector) is a separate effort.
|
||||
return await asyncio.to_thread(_score)
|
||||
|
||||
|
||||
async def backfill_note_embeddings() -> None:
|
||||
|
||||
@@ -140,6 +140,14 @@ async def _semantic_knowledge_search(
|
||||
Exact keyword matches always rank above semantic-only matches so that
|
||||
searching for a name like "Weston" surfaces the note with that title
|
||||
before conceptually related notes.
|
||||
|
||||
BEST-EFFORT TOP-N, not exhaustive pagination: the ranked candidate set is
|
||||
capped (keyword limit*2 + up to ~200 semantic), so `total` is the size of
|
||||
that window, NOT the true match count, and matches beyond the cap are not
|
||||
reachable by paging. Each page also recomputes the full merge (O(corpus)
|
||||
per page). Acceptable for an interactive "best results" feed; a cached
|
||||
ranked-id list or pgvector ORDER BY/LIMIT is the fix if exhaustive,
|
||||
cheap pagination is ever needed.
|
||||
"""
|
||||
# 1. Keyword search — title and body ILIKE
|
||||
keyword_notes: list[Note] = []
|
||||
|
||||
Reference in New Issue
Block a user