Upgrade intent model to qwen2.5:7b, simplify intent prompt rules
config.py: - Default OLLAMA_INTENT_MODEL: qwen2.5:1.5b → qwen2.5:7b - Startup will auto-pull and warm the new model on next container restart intent.py: - Replaced phrase-matching examples in search_web and research_topic rules with semantic descriptions. The 7B model doesn't need example phrases to understand intent — it can reason from the tool's purpose. Removes implied usage patterns that caused misclassifications on conversational phrasing (e.g. "I've been thinking about buying shirts, can you research this?"). - research_topic rule now explicitly covers any subject regardless of phrasing, including shopping decisions, comparisons, how-things-work questions, etc. - search_web rule clarified as "short summary, no note" vs research_topic's "comprehensive written reference" The 1.5B model required prescriptive phrase examples to route correctly; the 7B model has sufficient language understanding to classify from semantic intent. Expected improvement: ~1-2s intent calls (vs 0.4-9s for the 1.5B model which sometimes timed out or misclassified longer/conversational messages). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
@@ -24,9 +24,9 @@ class Config:
|
||||
)
|
||||
OLLAMA_URL: str = os.environ.get("OLLAMA_URL", "http://localhost:11434")
|
||||
OLLAMA_MODEL: str = os.environ.get("OLLAMA_MODEL", "qwen3:latest")
|
||||
# Optional dedicated model for intent classification (should be small/fast).
|
||||
# Optional dedicated model for intent classification.
|
||||
# Falls back to OLLAMA_MODEL if not set. Can also be overridden per-user via settings.
|
||||
OLLAMA_INTENT_MODEL: str = os.environ.get("OLLAMA_INTENT_MODEL", "qwen2.5:1.5b")
|
||||
OLLAMA_INTENT_MODEL: str = os.environ.get("OLLAMA_INTENT_MODEL", "qwen2.5:7b")
|
||||
# KV cache context window for generation. Lower = less VRAM, less throughput impact.
|
||||
# 8192 is sufficient for most conversations; raise if you paste large documents.
|
||||
OLLAMA_NUM_CTX: int = int(os.environ.get("OLLAMA_NUM_CTX", "8192"))
|
||||
|
||||
Reference in New Issue
Block a user