perf(retrieval): one embedding per query per plugin request (milestone 456 step 1, #4903)
CI & Build / Plugin hooks (push) Successful in 17s
CI & Build / Python lint (push) Successful in 2s
CI & Build / TypeScript typecheck (push) Successful in 55s
CI & Build / integration (push) Successful in 1m25s
CI & Build / Python tests (push) Successful in 2m6s
CI & Build / Build & push image (push) Successful in 1m25s

A single plugin request fans one query out to several ranked arms. The
operator message is searched by auto_inject, its reuse and lesson slots,
prompt_rule, the preference slot and rule_via_lesson, and each arm called
get_embedding on its own: up to six model calls for one vector.

- embeddings.query_embedding_memo(): a request-scoped ContextVar memo that
  get_embedding consults. Outside a scope the default is None, so every
  other caller is unchanged. A failed embedding is not remembered.
- memoized_query_embeddings decorates /retrieve, /tool-rules and
  /prior-art.
- get_embeddings (document chunks) is untouched.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
2026-10-05 07:55:06 -04:00
co-authored by Claude Opus 5.5
parent d2ac7bf220
commit be62abf142
3 changed files with 162 additions and 1 deletions
+4
View File
@@ -18,6 +18,7 @@ from scribe.services import repo_bindings as repo_bindings_svc
from scribe.services import report_check as report_check_svc
from scribe.services import shape_check as shape_check_svc
from scribe.services import task_claims as task_claims_svc
from scribe.services.embeddings import memoized_query_embeddings
from scribe.services.settings import get_admin_setting, set_setting
plugin_bp = Blueprint("plugin", __name__, url_prefix="/api/plugin")
@@ -88,6 +89,7 @@ async def session_context():
@plugin_bp.get("/retrieve")
@login_required
@memoized_query_embeddings
async def autoinject_retrieve():
"""Title-first knowledge auto-inject for the plugin's UserPromptSubmit hook.
@@ -163,6 +165,7 @@ async def autoinject_retrieve():
@plugin_bp.get("/tool-rules")
@login_required
@memoized_query_embeddings
async def pre_tool_rules():
"""Standing rules for the plugin's PreToolUse hook on ACTIONS (#3476).
@@ -240,6 +243,7 @@ async def pre_tool_rules():
@plugin_bp.get("/prior-art")
@login_required
@memoized_query_embeddings
async def write_path_prior_art():
"""Prior-art hint for the plugin's PreToolUse hook on Write/Edit.