ux: rename model fields + enforce serial curator execution
Three coordinated changes per operator request 2026-05-24:
1. Settings UI rename matching the language we actually use:
- Chat Model -> Chat & Voice Model
- Worker Model -> Curator Model
Setting KEYS (default_model / background_model) unchanged on
purpose; renaming them requires a migration touching 50+ call
sites for purely UX-facing benefit.
2. Settings UI help text rewritten:
- Chat & Voice: documents that it handles chat AND small
conversational automations (titles, tags). Recommends
OLLAMA_NUM_PARALLEL=2+ on the Ollama server so background
automations get their own KV-cache slot and don't evict
the chat model's working state.
- Curator: notes the app enforces SERIAL execution regardless
of NUM_PARALLEL — only one curator pass runs at a time. This
matters most for 70b CPU models where a second instance
would waste system RAM.
3. Enforce serial curator execution globally:
- New module-level _CURATOR_RUN_LOCK in services/curator.py.
- run_curator_for_conversation now wraps its body in 'async
with _CURATOR_RUN_LOCK' — every entry point (scheduler sweep,
manual route trigger, future hooks) is serialized through it.
- is_curator_running() helper exposes the lock state.
- routes/journal.py manual trigger checks is_curator_running()
first and returns 409 {busy: true} immediately rather than
blocking the HTTP request for minutes waiting for a 70b CPU
pass to finish. The user can retry once the curator clears.
Why a 409 instead of queue: a curator pass on a 70b CPU model
can take 5+ minutes. Tying up an HTTP worker that long is bad;
making the user wait without feedback is worse. 409 surfaces
the busy state immediately and the user retries when they want.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -1535,28 +1535,32 @@ function formatUserDate(iso: string): string {
|
||||
<p class="field-hint">The name used in chat messages and LLM context.</p>
|
||||
</div>
|
||||
<div class="field">
|
||||
<label for="default-model">Chat Model</label>
|
||||
<label for="default-model">Chat & Voice Model</label>
|
||||
<select id="default-model" v-model="defaultModel" class="input">
|
||||
<option value="">Default ({{ defaultChatModel || "qwen3:latest" }})</option>
|
||||
<option v-for="m in installedModels" :key="m" :value="m">{{ m }}</option>
|
||||
</select>
|
||||
<p class="field-hint">
|
||||
Used for journal chat (no tools — just conversation) and lightweight one-shot tasks like note-title generation and tag suggestions.
|
||||
Pick a small fast model (e.g. <code>qwen3:8b</code>, <code>llama3.2:3b</code>) — speed matters more than depth here.
|
||||
Powers the journal conversation (typed or voice-driven) AND small conversational automations: note-title generation, tag suggestions.
|
||||
Pick a small fast model (e.g. <code>qwen3:8b</code>, <code>llama3.2:3b</code>) — speed and responsiveness matter more than depth.
|
||||
Ideally runs on GPU.
|
||||
<br><br>
|
||||
<strong>Tip for snappy chat:</strong> set <code>OLLAMA_NUM_PARALLEL=2</code> (or higher) on your Ollama server so background automations (tags, titles) get their own KV-cache slot and don't evict the chat model's working state. With <code>NUM_PARALLEL=1</code>, every background call pauses the chat and re-warms the prompt on the next user turn.
|
||||
</p>
|
||||
</div>
|
||||
<div class="field">
|
||||
<label for="background-model">Worker Model</label>
|
||||
<label for="background-model">Curator Model</label>
|
||||
<select id="background-model" v-model="backgroundModel" class="input">
|
||||
<option value="">Default (qwen3:latest)</option>
|
||||
<option v-for="m in installedModels" :key="m" :value="m">{{ m }}</option>
|
||||
</select>
|
||||
<p class="field-hint">
|
||||
Used for heavy async work: the journal curator (capture / propose updates), daily prep generation, end-of-day closeout, task body consolidation, project summaries, and profile observation processing.
|
||||
Pick a smart model — latency doesn't matter, quality does. Often runs on CPU with system RAM (e.g. <code>qwen3:32b</code>, <code>qwen3:30b-a3b</code>).
|
||||
Powers the journal curator (capture moments, propose updates) and other heavy async work: daily prep generation, end-of-day closeout, task body consolidation, project summaries, profile observation processing.
|
||||
Pick a smart model — latency doesn't matter, quality does. Often runs on CPU with system RAM (e.g. <code>qwen3:32b</code>, <code>qwen3:30b-a3b</code>, <code>llama3.1:70b</code>).
|
||||
<br><br>
|
||||
<strong>Serialized:</strong> the app enforces one curator pass at a time globally, regardless of <code>OLLAMA_NUM_PARALLEL</code>. Manual triggers fired while the curator is busy return a "try again" response rather than spawning a second instance. This matters most for large CPU models where a second KV-cache slot would waste system RAM.
|
||||
<span v-if="backgroundModel && backgroundModel === (defaultModel || defaultChatModel)" class="field-hint-warn">
|
||||
⚠ Using the same model for both means worker tasks compete for the chat model's resources. Pick different models so they can be loaded simultaneously (<code>OLLAMA_MAX_LOADED_MODELS = 2</code>+).
|
||||
⚠ Using the same model for both means curator passes compete with chat for the same KV cache. Pick different models so both can stay loaded simultaneously (<code>OLLAMA_MAX_LOADED_MODELS = 2</code>+).
|
||||
</span>
|
||||
</p>
|
||||
</div>
|
||||
|
||||
@@ -199,7 +199,19 @@ async def trigger_curator_run(conv_id: int):
|
||||
if _res.scalar_one_or_none() is None:
|
||||
return jsonify({"error": "Conversation not found"}), 404
|
||||
|
||||
from fabledassistant.services.curator import run_curator_for_conversation
|
||||
from fabledassistant.services.curator import (
|
||||
is_curator_running,
|
||||
run_curator_for_conversation,
|
||||
)
|
||||
# The curator typically runs on a large model (30b-70b on CPU); we
|
||||
# serialize runs globally via a module-level lock. Reject rather
|
||||
# than block when busy — blocking would tie up an HTTP worker for
|
||||
# minutes. The user can retry in a moment.
|
||||
if is_curator_running():
|
||||
return jsonify({
|
||||
"error": "Curator is currently running. Please try again in a moment.",
|
||||
"busy": True,
|
||||
}), 409
|
||||
result = await run_curator_for_conversation(conv_id)
|
||||
|
||||
# Stamp last_curator_run_at on success so the scheduler doesn't
|
||||
|
||||
@@ -20,6 +20,7 @@ inline observer voice).
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import json
|
||||
import logging
|
||||
import time
|
||||
@@ -42,6 +43,28 @@ logger = logging.getLogger(__name__)
|
||||
# fixed transcript, not respond to evolving conversation).
|
||||
_MAX_TOOL_ROUNDS = 4
|
||||
|
||||
# Module-level serial lock for `run_curator_for_conversation`. The curator
|
||||
# model is typically large (30b-70b) and runs on CPU+RAM; loading more
|
||||
# than one curator request at a time wastes memory on a KV cache slot
|
||||
# that's never going to be used in parallel, and can swap the worker
|
||||
# model into thrashing. Every entry point — scheduler sweep, manual
|
||||
# trigger from the journal route, future hooks — must acquire this lock
|
||||
# so the system guarantees at most one curator pass runs at a time
|
||||
# globally. Manual-trigger route checks `is_curator_running()` first
|
||||
# and returns 409 rather than blocking the HTTP request.
|
||||
_CURATOR_RUN_LOCK = asyncio.Lock()
|
||||
|
||||
|
||||
def is_curator_running() -> bool:
|
||||
"""True iff a curator pass is currently executing.
|
||||
|
||||
Used by the manual-trigger route to decide between 'queue' (block on
|
||||
the lock) and 'reject' (return 409). Avoids tying up an HTTP request
|
||||
for minutes when a curator pass on a 70b CPU model is already in
|
||||
flight.
|
||||
"""
|
||||
return _CURATOR_RUN_LOCK.locked()
|
||||
|
||||
# Curator tool allowlist. Additive operations only — no updates, no
|
||||
# deletes. Risk model: the curator can be confidently wrong, and the
|
||||
# user is not in the loop when it runs. Adds are easily undone by the
|
||||
@@ -253,7 +276,27 @@ async def run_curator_for_conversation(
|
||||
the conversation row.
|
||||
|
||||
Returns a CuratorRunResult; never raises (errors land in result.error).
|
||||
|
||||
Guarded by `_CURATOR_RUN_LOCK` so at most one curator pass runs at
|
||||
once globally. The scheduler processes candidates serially within
|
||||
a sweep anyway; the lock matters when the manual-trigger route
|
||||
fires concurrently with a sweep (or vice versa). Manual-trigger
|
||||
callers should `is_curator_running()` first and reject rather than
|
||||
block, since acquiring this lock can take minutes for a large model.
|
||||
"""
|
||||
async with _CURATOR_RUN_LOCK:
|
||||
return await _run_curator_inner(
|
||||
conv_id, since=since, user_id_override=user_id_override,
|
||||
)
|
||||
|
||||
|
||||
async def _run_curator_inner(
|
||||
conv_id: int,
|
||||
*,
|
||||
since: datetime | None = None,
|
||||
user_id_override: int | None = None,
|
||||
) -> CuratorRunResult:
|
||||
"""Curator-pass body. Always invoked under `_CURATOR_RUN_LOCK`."""
|
||||
started_at = time.monotonic()
|
||||
|
||||
async with async_session() as session:
|
||||
|
||||
Reference in New Issue
Block a user