fix(telemetry): the two invisible retrievals log, and /api/search takes scope
CI & Build / Python lint (push) Successful in 3s
CI & Build / Plugin hooks (push) Successful in 9s
CI & Build / integration (push) Successful in 18s
CI & Build / TypeScript typecheck (push) Successful in 22s
CI & Build / Python tests (push) Failing after 28s
CI & Build / Build & push image (push) Skipped

#2463, the remaining findings (finding 3 landed with f11a547).

## The reuse slot logs (source: reuse_slot)

A real semantic query competing for an auto-inject menu slot, and the ledger
was asymmetric: the scored hit it DISPLACED was in retrieval_logs, the query
that displaced it was not — so the slot could never be evaluated against what
it replaced. #1038 and #2085 are gated on this ledger being complete.

## Browse search logs (source: browse_search)

The human's main search surface, and it logged nothing — so retrieval_logs
claimed the web's search was /api/search. Measured while fixing: /api/search
has ZERO frontend consumers; the web UI searches through /api/knowledge
exclusively. The table wasn't just under-describing the UI, it was describing
a surface the UI never touches.

The task's check — does folding human queries into the corpus skew the
precision signal thresholds are tuned against? — is answered by the source
column: distinct values (browse_search, reuse_slot beside the existing four)
mean tuning includes or excludes human traffic deliberately rather than by
accident. Same resolution as the mcp_/rest_ split in note_usage_events.

## /api/search takes project_id (and logs it)

The route logged project_id=None unconditionally while the MCP tool insists
the agent pass the active project. Now optional, default global: with no
frontend consumer, this route serves API callers, and an API caller states
its scope explicitly — the default-scope UI decision the task flagged is moot
until a UI actually consumes the route, which is recorded rather than guessed.

Refs #2463
This commit is contained in:
2026-08-08 20:00:22 -04:00
parent f11a547cd2
commit 45ba4aab25
3 changed files with 33 additions and 5 deletions
+7 -5
View File
@@ -36,22 +36,24 @@ async def search_route():
content_type = request.args.get("content_type", "all")
limit = min(request.args.get("limit", 10, type=int), 50)
is_task = _content_type_to_is_task(content_type)
# Same association filter the MCP tool takes (#33). The project filter this
# route is still missing is #2463's — it carries a default-scope UI decision
# this change must not preempt.
# Same association filters the MCP tool takes (#33). Optional, default
# global: this route has NO frontend consumer today (measured 2026-08-08 —
# the web UI searches through /api/knowledge), so it serves API callers,
# and an API caller states its scope explicitly.
system_id = request.args.get("system_id", type=int)
project_id = request.args.get("project_id", type=int)
t0 = time.perf_counter()
results = await semantic_search_notes(
uid, q, limit=limit, is_task=is_task, threshold=_REST_SEARCH_THRESHOLD,
system_id=system_id,
project_id=project_id, system_id=system_id,
# The user typed this, so it reaches everything they may read.
scope="read",
)
record_retrieval(
user_id=uid, source="rest_search", query=q,
threshold=_REST_SEARCH_THRESHOLD, limit=limit,
project_id=None, is_task=is_task, results=results,
project_id=project_id, is_task=is_task, results=results,
duration_ms=(time.perf_counter() - t0) * 1000.0,
)
owners = await owner_names_for(
+14
View File
@@ -401,6 +401,8 @@ async def _semantic_knowledge_search(
semantic_search_notes,
)
is_task_filter = True if note_type in ("task", "plan") else (False if note_type else None)
import time as _time
_t0 = _time.perf_counter()
candidates = await semantic_search_notes(
user_id=user_id,
scope="read",
@@ -413,6 +415,18 @@ async def _semantic_knowledge_search(
is_task=is_task_filter,
project_id=project_id,
)
# The human's MAIN search surface, and it logged nothing — so
# retrieval_logs claimed the web's search was /api/search, a narrower
# path with (measured) zero frontend consumers. Distinct source, so
# threshold tuning can include or exclude human queries deliberately
# rather than by accident (#2463).
from scribe.services.retrieval_telemetry import record_retrieval
record_retrieval(
user_id=user_id, source="browse_search", query=q,
threshold=INTERACTIVE_SEARCH_THRESHOLD, limit=min(200, limit * 4),
project_id=project_id, is_task=is_task_filter, results=candidates,
duration_ms=(_time.perf_counter() - _t0) * 1000.0,
)
for _score, note in candidates:
if note.deleted_at is not None:
continue
+12
View File
@@ -326,6 +326,7 @@ async def _reserve_slot_for_reuse(
return kept # reuse already represented; nothing to do
top_k = cfg["top_k"]
_t0 = time.perf_counter()
reuse = await semantic_search_notes(
user_id, query,
limit=1,
@@ -335,6 +336,17 @@ async def _reserve_slot_for_reuse(
note_type=_REUSE_KINDS,
scope="browse",
)
# A real semantic query competing for a menu slot — logged like the scored
# arm it displaces. Before this, the hit it PUSHED OUT was in
# retrieval_logs and the query that pushed it out was not, so the slot
# could never be evaluated against what it replaced (#2463; #1038 and
# #2085 are gated on this ledger being complete).
record_retrieval(
user_id=user_id, source="reuse_slot", query=query,
threshold=cfg["threshold"], limit=1, project_id=project_id,
is_task=None, results=reuse,
duration_ms=(time.perf_counter() - _t0) * 1000.0,
)
# Verify the kind rather than trusting the query that asked for it, and
# dedup on top of exclude_ids. This slot exists FOR reuse kinds — a slot
# silently spent on something else is worse than no slot, because the line