bvandeusenandClaude Opus 5
d5ac8408f6
CI & Build / Python lint (push) Successful in 8s
CI & Build / Plugin hooks (push) Successful in 16s
CI & Build / integration (push) Successful in 40s
CI & Build / TypeScript typecheck (push) Successful in 43s
CI & Build / Python tests (push) Successful in 1m14s
CI & Build / Build & push image (push) Successful in 2m59s
#3670 added `best_available_score` so a threshold could be judged from its rejections. It records how CLOSE the bar came to firing and not WHAT it refused, and that is the half a decision actually needs. Live, pre_tool_rule sits at a ~0.72 bar with a near-miss p90 of 0.7071 — about 117 declines a day within 0.013 of firing. Dropping to 0.707 would take that arm from 22 hits a day to roughly 139: six-fold, on a surface that runs before every Bash call. The percentile says the mass is there. Nothing said whether it was worth showing. NEITHER OBVIOUS INSTRUMENT ANSWERS IT. Pull-through cannot: the injected rule line already carries title and trigger, so a session can comply without ever calling get_rule, and rule pull-through understates usefulness by construction. Reading the rejected records can — and `result_ids` holds only what was RETURNED, so on a zero-result call the near-missed record had no name at all. So the id, from the SAME ranked candidate as the score. Both searches unpack `best` once and read both fields off it, because splitting that into two expressions is exactly how a later edit pairs a score with its neighbour's id — and a score attached to the wrong record is worse than no id, since it invites judging the wrong one and concluding the bar is fine. write_path withholds the id on the same condition it withholds the score (#3739): a surviving id beside a null score names a record without saying what it scored, the pair disagreeing in the other direction. THE READ PATH IS A LISTING, NOT A STATISTIC — an id cannot be percentiled, and a reader tuning a bar needs to go and read the records. Opt-in via `near_miss_samples` (0-20, default 0) so the ordinary readout keeps its size, and deliberately NOT a window function: this module's one production outage was a grouped query Postgres rejected, swallowed by the broad except, every counter reading zero while the mocked tests passed (#2663). One flat ordered query, overfetched, bucketed in Python — the shape that lesson prescribes. Migration 0097, nullable and unbackfilled. Not a foreign key: the table spans record types and `source` says which, exactly as result_ids works. The integration guard pins the listing as PER SOURCE. A global LIMIT would let a noisy source eat the whole quota and leave the surface being tuned showing nothing — which reads as "nothing was close", the misreading this milestone has spent itself correcting. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ
85 lines
3.2 KiB
Python
85 lines
3.2 KiB
Python
import time
|
|
|
|
from quart import Blueprint, jsonify, request
|
|
|
|
from scribe.auth import login_required, get_current_user_id
|
|
from scribe.services.access import owner_names_for
|
|
from scribe.services.embeddings import (
|
|
INTERACTIVE_SEARCH_THRESHOLD as _REST_SEARCH_THRESHOLD,
|
|
)
|
|
from scribe.services.embeddings import semantic_search_notes
|
|
from scribe.services.retrieval_telemetry import record_retrieval
|
|
|
|
# The interactive floor lives in embeddings.py now, shared with Browse search
|
|
# and the list views' semantic `q` — one number, one rationale (#2463).
|
|
|
|
search_bp = Blueprint("search", __name__, url_prefix="/api/search")
|
|
|
|
|
|
def _content_type_to_is_task(content_type: str) -> bool | None:
|
|
"""Map content_type query param to semantic_search_notes is_task arg."""
|
|
if content_type == "note":
|
|
return False
|
|
if content_type == "task":
|
|
return True
|
|
return None # "all" or unknown → no filter
|
|
|
|
|
|
@search_bp.route("", methods=["GET"])
|
|
@login_required
|
|
async def search_route():
|
|
uid = get_current_user_id()
|
|
q = (request.args.get("q") or "").strip()
|
|
if not q:
|
|
return jsonify({"error": "q is required"}), 400
|
|
|
|
content_type = request.args.get("content_type", "all")
|
|
limit = min(request.args.get("limit", 10, type=int), 50)
|
|
is_task = _content_type_to_is_task(content_type)
|
|
# Same association filters the MCP tool takes (#33). Optional, default
|
|
# global: this route has NO frontend consumer today (measured 2026-08-08 —
|
|
# the web UI searches through /api/knowledge), so it serves API callers,
|
|
# and an API caller states its scope explicitly.
|
|
system_id = request.args.get("system_id", type=int)
|
|
project_id = request.args.get("project_id", type=int)
|
|
|
|
t0 = time.perf_counter()
|
|
report: dict = {}
|
|
results = await semantic_search_notes(
|
|
uid, q, limit=limit, is_task=is_task, threshold=_REST_SEARCH_THRESHOLD,
|
|
project_id=project_id, system_id=system_id,
|
|
# The user typed this, so it reaches everything they may read.
|
|
scope="read",
|
|
report=report,
|
|
)
|
|
record_retrieval(
|
|
user_id=uid, source="rest_search", query=q,
|
|
threshold=_REST_SEARCH_THRESHOLD, limit=limit,
|
|
project_id=project_id, is_task=is_task, results=results,
|
|
duration_ms=(time.perf_counter() - t0) * 1000.0,
|
|
best_available=report.get("best_available_score"),
|
|
best_available_id=report.get("best_available_id"),
|
|
searched=bool(report.get("searched", True)),
|
|
)
|
|
owners = await owner_names_for(
|
|
{int(note.user_id) for _s, note in results if note.user_id != uid}
|
|
)
|
|
return jsonify({
|
|
"results": [
|
|
{
|
|
"id": note.id,
|
|
"title": note.title,
|
|
"body": note.body or "",
|
|
"is_task": note.is_task,
|
|
"tags": note.tags or [],
|
|
"similarity": score,
|
|
**(
|
|
{"shared": True, "owner": owners.get(int(note.user_id))}
|
|
if note.user_id != uid else {}
|
|
),
|
|
}
|
|
for score, note in results # semantic_search_notes returns list[tuple[float, Note]]
|
|
],
|
|
"total": len(results),
|
|
})
|