Phase 21: Intent-first pipeline, visible ack, KV-stable system prompt
Pipeline changes (generation_task.py, intent.py): - Remove optimistic streaming queue/race (_drain_queue deleted) - Remove _generate_acknowledgment — ack now embedded in intent JSON - Round 0: await intent (~400ms), stream ack immediately as TTFT, then execute tool sequentially; chat-only streams directly - IntentResult.ack: one-sentence acknowledgment, intent max_tokens 200→350 - _parse_intent extracts and trims ack field KV cache stability (llm.py, generation_buffer.py, generation_task.py): - build_context: replace cached_note_ids with include_note_ids - Auto-found notes populate context_meta["auto_notes"] for sidebar but are NOT injected into system prompt (--- Related Notes --- removed) - Explicitly included notes injected as --- Included Notes --- - _conv_note_cache dict + get/set/clear functions removed from generation_buffer.py - All clear_conv_note_cache() calls removed Cold model retry (llm.py): - generate_completion (used by classify_intent) retries on HTTP 500: 3 attempts with 3s/6s delays — prevents intent failure during cold load API + frontend (routes/chat.py, stores/chat.ts, views/ChatView.vue, components/ChatPanel.vue): - exclude_note_ids → include_note_ids throughout - ChatView sidebar: Suggested (auto-found, + to include) + In Context (× to remove) - ChatPanel: remove exclude button from context pills; no IDs passed to sendMessage Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
@@ -75,26 +75,6 @@ class GenerationBuffer:
|
||||
# Module-level singleton registry
|
||||
_buffers: dict[int | str, GenerationBuffer] = {}
|
||||
|
||||
# Per-conversation note context cache — maps conv_id → sorted list of note IDs.
|
||||
# Stores the note IDs that were last included in the system prompt so that
|
||||
# subsequent turns in the same conversation can reuse them, stabilizing the
|
||||
# system prompt prefix and improving Ollama's KV cache hit rate.
|
||||
_conv_note_cache: dict[int, list[int]] = {}
|
||||
|
||||
|
||||
def get_conv_note_cache(conv_id: int) -> list[int]:
|
||||
"""Return cached note IDs for a conversation (empty list if none)."""
|
||||
return list(_conv_note_cache.get(conv_id, []))
|
||||
|
||||
|
||||
def set_conv_note_cache(conv_id: int, note_ids: list[int]) -> None:
|
||||
"""Store note IDs to reuse on the next turn of this conversation."""
|
||||
_conv_note_cache[conv_id] = list(note_ids)
|
||||
|
||||
|
||||
def clear_conv_note_cache(conv_id: int) -> None:
|
||||
"""Invalidate the note cache for a conversation (e.g. after a note write)."""
|
||||
_conv_note_cache.pop(conv_id, None)
|
||||
_cleanup_task: asyncio.Task | None = None
|
||||
_GRACE_PERIOD = 60.0 # seconds to keep completed buffers
|
||||
|
||||
|
||||
Reference in New Issue
Block a user