test(rules): the scope test says why a rule search returned nothing - an error or an empty scan (#4958)
CI & Build / Python lint (push) Successful in 4s
CI & Build / Plugin hooks (push) Successful in 14s
CI & Build / TypeScript typecheck (push) Successful in 57s
CI & Build / integration (push) Successful in 1m6s
CI & Build / Python tests (push) Successful in 1m55s
CI & Build / Build & push image (push) Successful in 18s
CI & Build / Python lint (push) Successful in 4s
CI & Build / Plugin hooks (push) Successful in 14s
CI & Build / TypeScript typecheck (push) Successful in 57s
CI & Build / integration (push) Successful in 1m6s
CI & Build / Python tests (push) Successful in 1m55s
CI & Build / Build & push image (push) Successful in 18s
semantic_search_rules fails open, so both failures on run 8209 and main run 8212 read as set() with no trace. Assert report["searched"] and carry the swallowed traceback into the failure. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
@@ -7,6 +7,7 @@ join doing the scoping: these seed real rules with hand-made vectors and stub
|
|||||||
only the embedder, so every rule is an equally good match and the home alone
|
only the embedder, so every rule is an equally good match and the home alone
|
||||||
decides what comes back.
|
decides what comes back.
|
||||||
"""
|
"""
|
||||||
|
import traceback
|
||||||
import uuid
|
import uuid
|
||||||
from unittest.mock import AsyncMock, MagicMock, patch
|
from unittest.mock import AsyncMock, MagicMock, patch
|
||||||
|
|
||||||
@@ -74,10 +75,20 @@ async def homes():
|
|||||||
|
|
||||||
|
|
||||||
async def _found(user_id: int, **scope) -> set[int]:
|
async def _found(user_id: int, **scope) -> set[int]:
|
||||||
|
"""The rule ids a search finds — and a failure, not an empty set, when the
|
||||||
|
search never ran. `semantic_search_rules` fails open (an exception becomes
|
||||||
|
[]), so without `report["searched"]` an error and a scoping miss read the
|
||||||
|
same (#4958)."""
|
||||||
|
report: dict = {}
|
||||||
|
raised: list[str] = []
|
||||||
with patch("scribe.services.embeddings.get_embedding",
|
with patch("scribe.services.embeddings.get_embedding",
|
||||||
AsyncMock(return_value=QUERY_VEC)):
|
AsyncMock(return_value=QUERY_VEC)), \
|
||||||
|
patch("scribe.services.embeddings.logger") as log:
|
||||||
|
# Called inside the except block, so the traceback is still current.
|
||||||
|
log.warning.side_effect = lambda *_a, **_k: raised.append(traceback.format_exc())
|
||||||
hits = await semantic_search_rules(user_id, "anything", limit=10,
|
hits = await semantic_search_rules(user_id, "anything", limit=10,
|
||||||
threshold=0.5, **scope)
|
threshold=0.5, report=report, **scope)
|
||||||
|
assert report.get("searched"), "the rule search did not run:\n" + "\n".join(raised)
|
||||||
return {rule.id for _score, rule in hits}
|
return {rule.id for _score, rule in hits}
|
||||||
|
|
||||||
|
|
||||||
|
|||||||
Reference in New Issue
Block a user