perf(search): offload cosine scoring off event loop; document best-effort feed
Drift-audit Group 9 (param-cliff / unbounded search work): - semantic_search_notes: the O(rows) cosine-similarity scoring loop ran synchronously on the event loop, so every RAG injection / search stalled other requests proportional to the user's embedding count. Move the scoring into asyncio.to_thread (results unchanged). The deeper fix — bounding the candidate set via pgvector ORDER BY/LIMIT — is noted as separate infra work. - _semantic_knowledge_search: documented the best-effort top-N semantics — is the capped candidate-window size (not the true match count), matches beyond the cap aren't page-reachable, and each page recomputes the full merge. Prevents the silent-truncation trap; cached ranked-id paging / pgvector is the fix if exhaustive pagination is ever required. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -153,17 +153,22 @@ async def semantic_search_notes(
|
|||||||
if not rows:
|
if not rows:
|
||||||
return []
|
return []
|
||||||
|
|
||||||
scored: list[tuple[float, Note]] = []
|
def _score() -> list[tuple[float, Note]]:
|
||||||
for ne, note in rows:
|
out: list[tuple[float, Note]] = []
|
||||||
try:
|
for ne, note in rows:
|
||||||
sim = _cosine_similarity(query_vec, ne.embedding)
|
try:
|
||||||
except Exception:
|
sim = _cosine_similarity(query_vec, ne.embedding)
|
||||||
continue
|
except Exception:
|
||||||
if sim >= threshold:
|
continue
|
||||||
scored.append((sim, note))
|
if sim >= threshold:
|
||||||
|
out.append((sim, note))
|
||||||
|
out.sort(key=lambda x: x[0], reverse=True)
|
||||||
|
return out[:limit]
|
||||||
|
|
||||||
scored.sort(key=lambda x: x[0], reverse=True)
|
# Offload the O(rows) cosine scoring off the event loop so a large corpus
|
||||||
return scored[:limit]
|
# doesn't stall other requests while ranking. Results are unchanged; the
|
||||||
|
# real scaling fix (ORDER BY / LIMIT in pgvector) is a separate effort.
|
||||||
|
return await asyncio.to_thread(_score)
|
||||||
|
|
||||||
|
|
||||||
async def backfill_note_embeddings() -> None:
|
async def backfill_note_embeddings() -> None:
|
||||||
|
|||||||
@@ -140,6 +140,14 @@ async def _semantic_knowledge_search(
|
|||||||
Exact keyword matches always rank above semantic-only matches so that
|
Exact keyword matches always rank above semantic-only matches so that
|
||||||
searching for a name like "Weston" surfaces the note with that title
|
searching for a name like "Weston" surfaces the note with that title
|
||||||
before conceptually related notes.
|
before conceptually related notes.
|
||||||
|
|
||||||
|
BEST-EFFORT TOP-N, not exhaustive pagination: the ranked candidate set is
|
||||||
|
capped (keyword limit*2 + up to ~200 semantic), so `total` is the size of
|
||||||
|
that window, NOT the true match count, and matches beyond the cap are not
|
||||||
|
reachable by paging. Each page also recomputes the full merge (O(corpus)
|
||||||
|
per page). Acceptable for an interactive "best results" feed; a cached
|
||||||
|
ranked-id list or pgvector ORDER BY/LIMIT is the fix if exhaustive,
|
||||||
|
cheap pagination is ever needed.
|
||||||
"""
|
"""
|
||||||
# 1. Keyword search — title and body ILIKE
|
# 1. Keyword search — title and body ILIKE
|
||||||
keyword_notes: list[Note] = []
|
keyword_notes: list[Note] = []
|
||||||
|
|||||||
Reference in New Issue
Block a user