fix(llm): switch default background model to gemma3:4b
qwen2.5:3b produced broken auto-summaries (misspellings, token repetition, hallucinated topics) — its synthesis ceiling is too low for free-form summarization. Gemma 3 4B is stronger on summarization at similar size and still fits comfortably alongside the main chat model in VRAM, so it preserves the KV-cache-separation strategy that keeps chat TTFT fast. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
@@ -27,7 +27,7 @@ class Config:
|
||||
# Lightweight model for background tasks (title generation, tag suggestions,
|
||||
# project summaries, RSS classification). Using a separate model keeps the
|
||||
# main model's KV cache intact between user messages, enabling prefix cache hits.
|
||||
OLLAMA_BACKGROUND_MODEL: str = os.environ.get("OLLAMA_BACKGROUND_MODEL", "qwen2.5:3b")
|
||||
OLLAMA_BACKGROUND_MODEL: str = os.environ.get("OLLAMA_BACKGROUND_MODEL", "gemma3:4b")
|
||||
# Ollama keep_alive — how long a model stays resident in VRAM after its last
|
||||
# request. Main model gets a longer window since it's used interactively;
|
||||
# the background model is called sporadically and doesn't need to camp VRAM.
|
||||
|
||||
Reference in New Issue
Block a user