refactor(models): route tasks to chat vs worker per new architecture
Chat and background model roles effectively swapped during the conversation+curator pivot, but call sites still used OLD routing. This commit re-routes each call to the model whose new role fits. Moved to background_model (worker — heavy, deliberate): - services/journal_prep.py: daily prep generation. - services/user_profile.py: observation consolidation. Moved to default_model (chat — small, fast): - services/chat.py save_response_as_note: note title generation. - services/tag_suggestions.py: tag suggestions. Already routed correctly (unchanged): curator, closeout, consolidation, project summaries, history summarization. SettingsView.vue: help text rewritten for both model fields to describe new roles. Background Model UI label renamed to Worker Model so the heavier role is visible from the picker. Warning copy updated to recommend OLLAMA_MAX_LOADED_MODELS=2+ so chat and worker can stay loaded simultaneously. Schema names default_model and background_model unchanged on purpose (renaming requires migration + touches ~50 call sites for UX-only gain). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -1540,19 +1540,23 @@ function formatUserDate(iso: string): string {
|
||||
<option value="">Default ({{ defaultChatModel || "qwen3:latest" }})</option>
|
||||
<option v-for="m in installedModels" :key="m" :value="m">{{ m }}</option>
|
||||
</select>
|
||||
<p class="field-hint">Model used for new conversations.</p>
|
||||
<p class="field-hint">
|
||||
Used for journal chat (no tools — just conversation) and lightweight one-shot tasks like note-title generation and tag suggestions.
|
||||
Pick a small fast model (e.g. <code>qwen3:8b</code>, <code>llama3.2:3b</code>) — speed matters more than depth here.
|
||||
Ideally runs on GPU.
|
||||
</p>
|
||||
</div>
|
||||
<div class="field">
|
||||
<label for="background-model">Background Model</label>
|
||||
<label for="background-model">Worker Model</label>
|
||||
<select id="background-model" v-model="backgroundModel" class="input">
|
||||
<option value="">Default (qwen2.5:0.5b)</option>
|
||||
<option value="">Default (qwen3:latest)</option>
|
||||
<option v-for="m in installedModels" :key="m" :value="m">{{ m }}</option>
|
||||
</select>
|
||||
<p class="field-hint">
|
||||
Model used for background tasks: title generation, tag suggestions, and project summaries.
|
||||
Using a small dedicated model (e.g. qwen2.5:0.5b) keeps the chat model's KV cache warm between messages, significantly reducing response time.
|
||||
Used for heavy async work: the journal curator (capture / propose updates), daily prep generation, end-of-day closeout, task body consolidation, project summaries, and profile observation processing.
|
||||
Pick a smart model — latency doesn't matter, quality does. Often runs on CPU with system RAM (e.g. <code>qwen3:32b</code>, <code>qwen3:30b-a3b</code>).
|
||||
<span v-if="backgroundModel && backgroundModel === (defaultModel || defaultChatModel)" class="field-hint-warn">
|
||||
⚠ Setting this to the same model as Chat Model will wipe the KV cache after every background task, increasing response latency.
|
||||
⚠ Using the same model for both means worker tasks compete for the chat model's resources. Pick different models so they can be loaded simultaneously (<code>OLLAMA_MAX_LOADED_MODELS = 2</code>+).
|
||||
</span>
|
||||
</p>
|
||||
</div>
|
||||
|
||||
Reference in New Issue
Block a user