test(rules): the scope test says why a rule search returned nothing - an error or an empty scan (#4958)
CI & Build / Python lint (push) Successful in 4s
CI & Build / Plugin hooks (push) Successful in 14s
CI & Build / TypeScript typecheck (push) Successful in 57s
CI & Build / integration (push) Successful in 1m6s
CI & Build / Python tests (push) Successful in 1m55s
CI & Build / Build & push image (push) Successful in 18s
CI & Build / Python lint (push) Successful in 4s
CI & Build / Plugin hooks (push) Successful in 14s
CI & Build / TypeScript typecheck (push) Successful in 57s
CI & Build / integration (push) Successful in 1m6s
CI & Build / Python tests (push) Successful in 1m55s
CI & Build / Build & push image (push) Successful in 18s
semantic_search_rules fails open, so both failures on run 8209 and main run 8212 read as set() with no trace. Assert report["searched"] and carry the swallowed traceback into the failure. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
@@ -7,6 +7,7 @@ join doing the scoping: these seed real rules with hand-made vectors and stub
|
||||
only the embedder, so every rule is an equally good match and the home alone
|
||||
decides what comes back.
|
||||
"""
|
||||
import traceback
|
||||
import uuid
|
||||
from unittest.mock import AsyncMock, MagicMock, patch
|
||||
|
||||
@@ -74,10 +75,20 @@ async def homes():
|
||||
|
||||
|
||||
async def _found(user_id: int, **scope) -> set[int]:
|
||||
"""The rule ids a search finds — and a failure, not an empty set, when the
|
||||
search never ran. `semantic_search_rules` fails open (an exception becomes
|
||||
[]), so without `report["searched"]` an error and a scoping miss read the
|
||||
same (#4958)."""
|
||||
report: dict = {}
|
||||
raised: list[str] = []
|
||||
with patch("scribe.services.embeddings.get_embedding",
|
||||
AsyncMock(return_value=QUERY_VEC)):
|
||||
AsyncMock(return_value=QUERY_VEC)), \
|
||||
patch("scribe.services.embeddings.logger") as log:
|
||||
# Called inside the except block, so the traceback is still current.
|
||||
log.warning.side_effect = lambda *_a, **_k: raised.append(traceback.format_exc())
|
||||
hits = await semantic_search_rules(user_id, "anything", limit=10,
|
||||
threshold=0.5, **scope)
|
||||
threshold=0.5, report=report, **scope)
|
||||
assert report.get("searched"), "the rule search did not run:\n" + "\n".join(raised)
|
||||
return {rule.id for _score, rule in hits}
|
||||
|
||||
|
||||
|
||||
Reference in New Issue
Block a user