fix(telemetry): the two invisible retrievals log, and /api/search takes scope
CI & Build / Python lint (push) Successful in 3s
CI & Build / Plugin hooks (push) Successful in 9s
CI & Build / integration (push) Successful in 18s
CI & Build / TypeScript typecheck (push) Successful in 22s
CI & Build / Python tests (push) Failing after 28s
CI & Build / Build & push image (push) Skipped
CI & Build / Python lint (push) Successful in 3s
CI & Build / Plugin hooks (push) Successful in 9s
CI & Build / integration (push) Successful in 18s
CI & Build / TypeScript typecheck (push) Successful in 22s
CI & Build / Python tests (push) Failing after 28s
CI & Build / Build & push image (push) Skipped
#2463, the remaining findings (finding 3 landed with f11a547).
## The reuse slot logs (source: reuse_slot)
A real semantic query competing for an auto-inject menu slot, and the ledger
was asymmetric: the scored hit it DISPLACED was in retrieval_logs, the query
that displaced it was not — so the slot could never be evaluated against what
it replaced. #1038 and #2085 are gated on this ledger being complete.
## Browse search logs (source: browse_search)
The human's main search surface, and it logged nothing — so retrieval_logs
claimed the web's search was /api/search. Measured while fixing: /api/search
has ZERO frontend consumers; the web UI searches through /api/knowledge
exclusively. The table wasn't just under-describing the UI, it was describing
a surface the UI never touches.
The task's check — does folding human queries into the corpus skew the
precision signal thresholds are tuned against? — is answered by the source
column: distinct values (browse_search, reuse_slot beside the existing four)
mean tuning includes or excludes human traffic deliberately rather than by
accident. Same resolution as the mcp_/rest_ split in note_usage_events.
## /api/search takes project_id (and logs it)
The route logged project_id=None unconditionally while the MCP tool insists
the agent pass the active project. Now optional, default global: with no
frontend consumer, this route serves API callers, and an API caller states
its scope explicitly — the default-scope UI decision the task flagged is moot
until a UI actually consumes the route, which is recorded rather than guessed.
Refs #2463
This commit is contained in:
@@ -36,22 +36,24 @@ async def search_route():
|
|||||||
content_type = request.args.get("content_type", "all")
|
content_type = request.args.get("content_type", "all")
|
||||||
limit = min(request.args.get("limit", 10, type=int), 50)
|
limit = min(request.args.get("limit", 10, type=int), 50)
|
||||||
is_task = _content_type_to_is_task(content_type)
|
is_task = _content_type_to_is_task(content_type)
|
||||||
# Same association filter the MCP tool takes (#33). The project filter this
|
# Same association filters the MCP tool takes (#33). Optional, default
|
||||||
# route is still missing is #2463's — it carries a default-scope UI decision
|
# global: this route has NO frontend consumer today (measured 2026-08-08 —
|
||||||
# this change must not preempt.
|
# the web UI searches through /api/knowledge), so it serves API callers,
|
||||||
|
# and an API caller states its scope explicitly.
|
||||||
system_id = request.args.get("system_id", type=int)
|
system_id = request.args.get("system_id", type=int)
|
||||||
|
project_id = request.args.get("project_id", type=int)
|
||||||
|
|
||||||
t0 = time.perf_counter()
|
t0 = time.perf_counter()
|
||||||
results = await semantic_search_notes(
|
results = await semantic_search_notes(
|
||||||
uid, q, limit=limit, is_task=is_task, threshold=_REST_SEARCH_THRESHOLD,
|
uid, q, limit=limit, is_task=is_task, threshold=_REST_SEARCH_THRESHOLD,
|
||||||
system_id=system_id,
|
project_id=project_id, system_id=system_id,
|
||||||
# The user typed this, so it reaches everything they may read.
|
# The user typed this, so it reaches everything they may read.
|
||||||
scope="read",
|
scope="read",
|
||||||
)
|
)
|
||||||
record_retrieval(
|
record_retrieval(
|
||||||
user_id=uid, source="rest_search", query=q,
|
user_id=uid, source="rest_search", query=q,
|
||||||
threshold=_REST_SEARCH_THRESHOLD, limit=limit,
|
threshold=_REST_SEARCH_THRESHOLD, limit=limit,
|
||||||
project_id=None, is_task=is_task, results=results,
|
project_id=project_id, is_task=is_task, results=results,
|
||||||
duration_ms=(time.perf_counter() - t0) * 1000.0,
|
duration_ms=(time.perf_counter() - t0) * 1000.0,
|
||||||
)
|
)
|
||||||
owners = await owner_names_for(
|
owners = await owner_names_for(
|
||||||
|
|||||||
@@ -401,6 +401,8 @@ async def _semantic_knowledge_search(
|
|||||||
semantic_search_notes,
|
semantic_search_notes,
|
||||||
)
|
)
|
||||||
is_task_filter = True if note_type in ("task", "plan") else (False if note_type else None)
|
is_task_filter = True if note_type in ("task", "plan") else (False if note_type else None)
|
||||||
|
import time as _time
|
||||||
|
_t0 = _time.perf_counter()
|
||||||
candidates = await semantic_search_notes(
|
candidates = await semantic_search_notes(
|
||||||
user_id=user_id,
|
user_id=user_id,
|
||||||
scope="read",
|
scope="read",
|
||||||
@@ -413,6 +415,18 @@ async def _semantic_knowledge_search(
|
|||||||
is_task=is_task_filter,
|
is_task=is_task_filter,
|
||||||
project_id=project_id,
|
project_id=project_id,
|
||||||
)
|
)
|
||||||
|
# The human's MAIN search surface, and it logged nothing — so
|
||||||
|
# retrieval_logs claimed the web's search was /api/search, a narrower
|
||||||
|
# path with (measured) zero frontend consumers. Distinct source, so
|
||||||
|
# threshold tuning can include or exclude human queries deliberately
|
||||||
|
# rather than by accident (#2463).
|
||||||
|
from scribe.services.retrieval_telemetry import record_retrieval
|
||||||
|
record_retrieval(
|
||||||
|
user_id=user_id, source="browse_search", query=q,
|
||||||
|
threshold=INTERACTIVE_SEARCH_THRESHOLD, limit=min(200, limit * 4),
|
||||||
|
project_id=project_id, is_task=is_task_filter, results=candidates,
|
||||||
|
duration_ms=(_time.perf_counter() - _t0) * 1000.0,
|
||||||
|
)
|
||||||
for _score, note in candidates:
|
for _score, note in candidates:
|
||||||
if note.deleted_at is not None:
|
if note.deleted_at is not None:
|
||||||
continue
|
continue
|
||||||
|
|||||||
@@ -326,6 +326,7 @@ async def _reserve_slot_for_reuse(
|
|||||||
return kept # reuse already represented; nothing to do
|
return kept # reuse already represented; nothing to do
|
||||||
|
|
||||||
top_k = cfg["top_k"]
|
top_k = cfg["top_k"]
|
||||||
|
_t0 = time.perf_counter()
|
||||||
reuse = await semantic_search_notes(
|
reuse = await semantic_search_notes(
|
||||||
user_id, query,
|
user_id, query,
|
||||||
limit=1,
|
limit=1,
|
||||||
@@ -335,6 +336,17 @@ async def _reserve_slot_for_reuse(
|
|||||||
note_type=_REUSE_KINDS,
|
note_type=_REUSE_KINDS,
|
||||||
scope="browse",
|
scope="browse",
|
||||||
)
|
)
|
||||||
|
# A real semantic query competing for a menu slot — logged like the scored
|
||||||
|
# arm it displaces. Before this, the hit it PUSHED OUT was in
|
||||||
|
# retrieval_logs and the query that pushed it out was not, so the slot
|
||||||
|
# could never be evaluated against what it replaced (#2463; #1038 and
|
||||||
|
# #2085 are gated on this ledger being complete).
|
||||||
|
record_retrieval(
|
||||||
|
user_id=user_id, source="reuse_slot", query=query,
|
||||||
|
threshold=cfg["threshold"], limit=1, project_id=project_id,
|
||||||
|
is_task=None, results=reuse,
|
||||||
|
duration_ms=(time.perf_counter() - _t0) * 1000.0,
|
||||||
|
)
|
||||||
# Verify the kind rather than trusting the query that asked for it, and
|
# Verify the kind rather than trusting the query that asked for it, and
|
||||||
# dedup on top of exclude_ids. This slot exists FOR reuse kinds — a slot
|
# dedup on top of exclude_ids. This slot exists FOR reuse kinds — a slot
|
||||||
# silently spent on something else is worse than no slot, because the line
|
# silently spent on something else is worse than no slot, because the line
|
||||||
|
|||||||
Reference in New Issue
Block a user