Phase 21: Intent-first pipeline, visible ack, KV-stable system prompt
Pipeline changes (generation_task.py, intent.py): - Remove optimistic streaming queue/race (_drain_queue deleted) - Remove _generate_acknowledgment — ack now embedded in intent JSON - Round 0: await intent (~400ms), stream ack immediately as TTFT, then execute tool sequentially; chat-only streams directly - IntentResult.ack: one-sentence acknowledgment, intent max_tokens 200→350 - _parse_intent extracts and trims ack field KV cache stability (llm.py, generation_buffer.py, generation_task.py): - build_context: replace cached_note_ids with include_note_ids - Auto-found notes populate context_meta["auto_notes"] for sidebar but are NOT injected into system prompt (--- Related Notes --- removed) - Explicitly included notes injected as --- Included Notes --- - _conv_note_cache dict + get/set/clear functions removed from generation_buffer.py - All clear_conv_note_cache() calls removed Cold model retry (llm.py): - generate_completion (used by classify_intent) retries on HTTP 500: 3 attempts with 3s/6s delays — prevents intent failure during cold load API + frontend (routes/chat.py, stores/chat.ts, views/ChatView.vue, components/ChatPanel.vue): - exclude_note_ids → include_note_ids throughout - ChatView sidebar: Suggested (auto-found, + to include) + In Context (× to remove) - ChatPanel: remove exclude button from context pills; no IDs passed to sendMessage Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
@@ -22,6 +22,7 @@ class IntentResult:
|
||||
tool_name: str | None = None # None = no tool, just chat
|
||||
arguments: dict = field(default_factory=dict)
|
||||
confidence: str = "high" # "high", "medium", or "low"
|
||||
ack: str | None = None # One-sentence acknowledgment to stream immediately
|
||||
|
||||
@property
|
||||
def should_execute(self) -> bool:
|
||||
@@ -61,8 +62,8 @@ Available tools:
|
||||
{tool_summary}
|
||||
|
||||
Respond with ONLY a JSON object, no other text:
|
||||
- If a tool should be called: {{"tool": "tool_name", "arguments": {{...}}, "confidence": "high"|"medium"|"low"}}
|
||||
- If it's general chat: {{"tool": null, "confidence": "high"}}
|
||||
- If a tool should be called: {{"tool": "tool_name", "arguments": {{...}}, "confidence": "high"|"medium"|"low", "ack": "One short sentence describing what you're about to do."}}
|
||||
- If it's general chat: {{"tool": null, "confidence": "high", "ack": null}}
|
||||
|
||||
Confidence levels:
|
||||
- "high": the intent is clear and all required arguments are unambiguous
|
||||
@@ -99,6 +100,7 @@ Rules:
|
||||
- "read", "open", "show me", "what does X say", "display", "pull up" a specific note → use get_note with query=<note name>.
|
||||
- "list my notes", "show notes", "recent notes", "browse notes", "notes tagged X" → use list_notes (with optional q or tags).
|
||||
- "tag X with Y", "add tag Y to X", "untag Y from X", "remove tag Y from X" → use update_note with tags=[Y] and tag_mode="add" or "remove".
|
||||
- "ack": one short, natural sentence confirming the action (tool path only). Vary phrasing — do not always start with "Let me". Omit (null) for chat-only responses.
|
||||
- Do NOT wrap the JSON in markdown code fences."""
|
||||
|
||||
|
||||
@@ -142,7 +144,7 @@ async def classify_intent(
|
||||
messages.append({"role": "user", "content": user_message})
|
||||
|
||||
try:
|
||||
raw = await generate_completion(messages, model, max_tokens=200)
|
||||
raw = await generate_completion(messages, model, max_tokens=350)
|
||||
except Exception:
|
||||
logger.warning("Intent classification LLM call failed", exc_info=True)
|
||||
return IntentResult()
|
||||
@@ -192,11 +194,15 @@ def _parse_intent(raw: str, tools: list[dict]) -> IntentResult:
|
||||
if not isinstance(arguments, dict):
|
||||
arguments = {}
|
||||
|
||||
ack = parsed.get("ack") or None
|
||||
if ack is not None:
|
||||
ack = ack.strip() or None
|
||||
|
||||
logger.info(
|
||||
"Intent classified: tool=%s, confidence=%s, args=%s",
|
||||
tool_name, confidence, json.dumps(arguments)[:200],
|
||||
)
|
||||
return IntentResult(tool_name=tool_name, arguments=arguments, confidence=confidence)
|
||||
return IntentResult(tool_name=tool_name, arguments=arguments, confidence=confidence, ack=ack)
|
||||
|
||||
|
||||
def _try_json(text: str) -> dict | list | None:
|
||||
|
||||
Reference in New Issue
Block a user