test(rules): the scope test says why a rule search returned nothing - an error or an empty scan (#4958)
CI & Build / Python lint (push) Successful in 4s
CI & Build / Plugin hooks (push) Successful in 14s
CI & Build / TypeScript typecheck (push) Successful in 57s
CI & Build / integration (push) Successful in 1m6s
CI & Build / Python tests (push) Successful in 1m55s
CI & Build / Build & push image (push) Successful in 18s

semantic_search_rules fails open, so both failures on run 8209 and main
run 8212 read as set() with no trace. Assert report["searched"] and carry
the swallowed traceback into the failure.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
2026-10-05 18:34:16 -04:00
co-authored by Claude Opus 5.5
parent 4e1320120d
commit 79cd2341a6
+13 -2
View File
@@ -7,6 +7,7 @@ join doing the scoping: these seed real rules with hand-made vectors and stub
only the embedder, so every rule is an equally good match and the home alone only the embedder, so every rule is an equally good match and the home alone
decides what comes back. decides what comes back.
""" """
import traceback
import uuid import uuid
from unittest.mock import AsyncMock, MagicMock, patch from unittest.mock import AsyncMock, MagicMock, patch
@@ -74,10 +75,20 @@ async def homes():
async def _found(user_id: int, **scope) -> set[int]: async def _found(user_id: int, **scope) -> set[int]:
"""The rule ids a search finds — and a failure, not an empty set, when the
search never ran. `semantic_search_rules` fails open (an exception becomes
[]), so without `report["searched"]` an error and a scoping miss read the
same (#4958)."""
report: dict = {}
raised: list[str] = []
with patch("scribe.services.embeddings.get_embedding", with patch("scribe.services.embeddings.get_embedding",
AsyncMock(return_value=QUERY_VEC)): AsyncMock(return_value=QUERY_VEC)), \
patch("scribe.services.embeddings.logger") as log:
# Called inside the except block, so the traceback is still current.
log.warning.side_effect = lambda *_a, **_k: raised.append(traceback.format_exc())
hits = await semantic_search_rules(user_id, "anything", limit=10, hits = await semantic_search_rules(user_id, "anything", limit=10,
threshold=0.5, **scope) threshold=0.5, report=report, **scope)
assert report.get("searched"), "the rule search did not run:\n" + "\n".join(raised)
return {rule.id for _score, rule in hits} return {rule.id for _score, rule in hits}