bvandeusen/FabledScribe
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 50 Wiki Activity

Add streaming status UX and model load state indicator

Browse Source 
Streaming status transparency:
- generation_task.py emits 'status' SSE events at each pipeline stage:
  "Analyzing your request..." before intent classification, tool label
  before each tool execution, "Generating/Composing response..." before
  each LLM streaming round
- chat.ts adds streamingStatus ref; cleared on first chunk or done/error;
  includes fast 5s poll loop after warmModel() until model shows as loaded
- ChatView.vue shows pulsing dot + italic status label above content area;
  falls back to blinking cursor once content arrives
- HomeView.vue shows status label in dashboard panel instead of '...'

Model load state indicator:
- /api/chat/status now queries /api/tags and /api/ps in parallel to
  distinguish installed-but-cold vs loaded-in-VRAM model states
- New model status values: 'not_found' | 'cold' | 'loaded' (was 'ready')
- chatReady true for both 'cold' and 'loaded' (cold models still work)
- AppHeader shows 5 states: gray pulse (checking), red (Ollama down),
  orange (not installed), yellow pulse (cold), green (loaded)
- Inline short label ("Cold", "Ready", "Offline", etc.) visible without
  hovering; detailed tooltip on hover

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
Bryan Van Deusen
2026-02-18 17:54:37 -05:00
parent b8acd05ec4
commit fbce540638
8 changed files with 171 additions and 24 deletions
Download Patch File Download Diff File Expand all files Collapse all files
frontend/src/components/AppHeader.vue
+37 -9
View File
@@ -14,18 +14,31 @@ const chatStore = useChatStore();
const router = useRouter();
const mobileMenuOpen = ref(false);
const statusShortLabel = computed(() => {
if (chatStore.ollamaStatus === "unavailable") return "Offline";
if (chatStore.ollamaStatus === "checking") return "Checking";
if (chatStore.modelStatus === "not_found") return "No model";
if (chatStore.modelStatus === "cold") return "Cold";
if (chatStore.modelStatus === "loaded") return "Ready";
return "Checking";
});
const statusLabel = computed(() => {
if (chatStore.ollamaStatus === "unavailable") return "Ollama unavailable";
if (chatStore.modelStatus === "not_found") return "Model downloading...";
if (chatStore.chatReady) return "Connected";
if (chatStore.ollamaStatus === "checking") return "Checking...";
if (chatStore.modelStatus === "not_found") return "Model not installed";
if (chatStore.modelStatus === "cold") return "Model cold — first response may be slow";
if (chatStore.modelStatus === "loaded") return "Model loaded · ready";
return "Checking...";
});
const statusClass = computed(() => {
if (chatStore.ollamaStatus === "unavailable") return "status-red";
if (chatStore.modelStatus === "not_found") return "status-yellow";
if (chatStore.chatReady) return "status-green";
return "status-yellow";
if (chatStore.ollamaStatus === "checking") return "status-gray";
if (chatStore.modelStatus === "not_found") return "status-orange";
if (chatStore.modelStatus === "cold") return "status-yellow";
if (chatStore.modelStatus === "loaded") return "status-green";
return "status-gray";
});
function toggleMobileMenu() {
@@ -67,6 +80,7 @@ router.afterEach(() => {
<router-link v-if="authStore.isAdmin" to="/admin/logs" class="nav-link">Logs</router-link>
<span class="status-indicator" :class="statusClass" :title="statusLabel">
<span class="status-dot"></span>
<span class="status-text">{{ statusShortLabel }}</span>
</span>
<button class="btn-shortcuts" @click="toggleShortcuts" title="Keyboard shortcuts (?)">
?
@@ -147,7 +161,9 @@ router.afterEach(() => {
.status-indicator {
display: flex;
align-items: center;
gap: 0.3rem;
margin-left: 0.5rem;
cursor: default;
}
.status-dot {
width: 8px;
@@ -155,19 +171,31 @@ router.afterEach(() => {
border-radius: 50%;
flex-shrink: 0;
}
.status-text {
font-size: 0.75rem;
font-weight: 500;
color: var(--color-text-muted);
}
.status-green .status-dot {
background: var(--color-success);
background: var(--color-success, #2ecc71);
}
.status-yellow .status-dot {
background: var(--color-warning);
background: var(--color-warning, #f59e0b);
animation: pulse-dot 2s infinite;
}
.status-orange .status-dot {
background: #f97316;
}
.status-red .status-dot {
background: var(--color-danger);
background: var(--color-danger, #e74c3c);
}
.status-gray .status-dot {
background: var(--color-text-muted);
animation: pulse-dot 2s infinite;
}
@keyframes pulse-dot {
0%, 100% { opacity: 1; }
50% { opacity: 0.4; }
50% { opacity: 0.3; }
}
.btn-shortcuts {
background: none;
frontend/src/stores/chat.ts
+26 -2
View File
@@ -26,14 +26,16 @@ export const useChatStore = defineStore("chat", () => {
const streaming = ref(false);
const streamingContent = ref("");
const streamingToolCalls = ref<ToolCallRecord[]>([]);
const streamingStatus = ref("");
const lastContextMeta = ref<ContextMeta | null>(null);
const ollamaStatus = ref<"checking" | "available" | "unavailable">("checking");
const modelStatus = ref<"checking" | "ready" | "not_found">("checking");
const modelStatus = ref<"checking" | "loaded" | "cold" | "not_found">("checking");
const defaultModel = ref("");
let statusPollTimer: ReturnType<typeof setInterval> | null = null;
// Chat is usable when Ollama is up and model is at least installed (cold starts still work)
const chatReady = computed(
() => ollamaStatus.value === "available" && modelStatus.value === "ready"
() => ollamaStatus.value === "available" && (modelStatus.value === "loaded" || modelStatus.value === "cold")
);
async function fetchConversations(limit = 50, offset = 0) {
@@ -178,8 +180,12 @@ export const useChatStore = defineStore("chat", () => {
case "context":
lastContextMeta.value = event.data.context as ContextMeta;
break;
case "status":
streamingStatus.value = event.data.status as string;
break;
case "chunk":
streamingContent.value += event.data.chunk as string;
streamingStatus.value = "";
break;
case "tool_call":
streamingToolCalls.value = [
@@ -206,6 +212,7 @@ export const useChatStore = defineStore("chat", () => {
}
streamingContent.value = "";
streamingToolCalls.value = [];
streamingStatus.value = "";
streaming.value = false;
// Update conversation in list
@@ -221,6 +228,7 @@ export const useChatStore = defineStore("chat", () => {
streaming.value = false;
streamingContent.value = "";
streamingToolCalls.value = [];
streamingStatus.value = "";
useToastStore().show(
"Chat error: " + (event.data.error as string),
"error",
@@ -258,6 +266,7 @@ export const useChatStore = defineStore("chat", () => {
streaming.value = false;
streamingContent.value = "";
streamingToolCalls.value = [];
streamingStatus.value = "";
}
async function cancelGeneration() {
@@ -314,11 +323,25 @@ export const useChatStore = defineStore("chat", () => {
async function warmModel(model: string) {
try {
await apiPost("/api/chat/warm", { model });
// Poll faster after a warm request so the indicator updates promptly
_pollUntilLoaded();
} catch {
// Warming is best-effort
}
}
function _pollUntilLoaded() {
let attempts = 0;
const MAX = 12; // up to ~60s of fast polling
const timer = setInterval(async () => {
await fetchStatus();
attempts++;
if (modelStatus.value === "loaded" || attempts >= MAX) {
clearInterval(timer);
}
}, 5_000);
}
return {
conversations,
currentConversation,
@@ -327,6 +350,7 @@ export const useChatStore = defineStore("chat", () => {
streaming,
streamingContent,
streamingToolCalls,
streamingStatus,
lastContextMeta,
ollamaStatus,
modelStatus,
frontend/src/types/chat.ts
+1 -1
View File
@@ -42,6 +42,6 @@ export interface ContextMeta {
export interface OllamaStatus {
ollama: "available" | "unavailable";
model: "ready" | "not_found";
model: "loaded" | "cold" | "not_found";
default_model: string;
}
frontend/src/views/ChatView.vue
+23 -1
View File
@@ -402,8 +402,12 @@ onUnmounted(() => {
:tool-call="tc"
/>
</div>
<div v-if="store.streamingStatus" class="streaming-status-line">
<span class="streaming-status-dot"></span>
{{ store.streamingStatus }}
</div>
<div class="message-content prose" v-html="streamingRendered"></div>
<span class="typing-indicator"></span>
<span v-if="!store.streamingStatus" class="typing-indicator"></span>
</div>
</div>
@@ -698,6 +702,24 @@ onUnmounted(() => {
gap: 0.3rem;
margin-bottom: 0.4rem;
}
.streaming-status-line {
display: flex;
align-items: center;
gap: 0.4rem;
font-size: 0.8rem;
color: var(--color-text-muted);
font-style: italic;
margin-bottom: 0.3rem;
}
.streaming-status-dot {
display: inline-block;
width: 6px;
height: 6px;
border-radius: 50%;
background: var(--color-primary);
animation: blink 1s infinite;
flex-shrink: 0;
}
.typing-indicator {
display: inline-block;
width: 6px;
frontend/src/views/HomeView.vue
+20 -1
View File
@@ -360,7 +360,11 @@ function clearDashboardResponse() {
<!-- Streaming text -->
<div v-if="chatStore.streaming" class="dashboard-response-text streaming">
<span v-if="chatStore.streamingContent">{{ chatStore.streamingContent }}</span>
<div v-if="chatStore.streamingStatus && !chatStore.streamingContent" class="dashboard-status-line">
<span class="dashboard-status-dot"></span>
{{ chatStore.streamingStatus }}
</div>
<span v-else-if="chatStore.streamingContent">{{ chatStore.streamingContent }}</span>
<span v-else class="thinking-dots">...</span>
</div>
<!-- Final text -->
@@ -553,6 +557,21 @@ function clearDashboardResponse() {
display: inline-block;
animation: blink 1.2s infinite;
}
.dashboard-status-line {
display: flex;
align-items: center;
gap: 0.4rem;
font-style: italic;
}
.dashboard-status-dot {
display: inline-block;
width: 6px;
height: 6px;
border-radius: 50%;
background: var(--color-primary);
animation: blink 1.2s infinite;
flex-shrink: 0;
}
@keyframes blink {
0%, 100% { opacity: 1; }
50% { opacity: 0.3; }
src/fabledassistant/routes/chat.py
+21 -9
View File
@@ -285,7 +285,7 @@ async def warm_model_route():
@chat_bp.route("/status", methods=["GET"])
@login_required
async def chat_status_route():
"""Check Ollama availability and model readiness."""
"""Check Ollama availability, model installation, and model load state."""
uid = get_current_user_id()
default_model = await get_setting(uid, "default_model", Config.OLLAMA_MODEL)
result = {
@@ -295,14 +295,26 @@ async def chat_status_route():
}
try:
async with httpx.AsyncClient(timeout=5.0) as client:
resp = await client.get(f"{Config.OLLAMA_URL}/api/tags")
resp.raise_for_status()
result["ollama"] = "available"
data = resp.json()
model_names = {m["name"] for m in data.get("models", [])}
# Match with or without :latest tag
if default_model in model_names or f"{default_model}:latest" in model_names:
result["model"] = "ready"
# Query installed models and loaded models in parallel
tags_task = asyncio.create_task(client.get(f"{Config.OLLAMA_URL}/api/tags"))
ps_task = asyncio.create_task(client.get(f"{Config.OLLAMA_URL}/api/ps"))
tags_resp, ps_resp = await asyncio.gather(tags_task, ps_task, return_exceptions=True)
if isinstance(tags_resp, Exception):
logger.debug("Ollama /api/tags failed: %s", tags_resp)
else:
tags_resp.raise_for_status()
result["ollama"] = "available"
model_names = {m["name"] for m in tags_resp.json().get("models", [])}
base = default_model.removesuffix(":latest")
if default_model in model_names or f"{base}:latest" in model_names:
# Installed — now check if currently loaded in memory
result["model"] = "cold"
if not isinstance(ps_resp, Exception):
ps_resp.raise_for_status()
loaded_names = {m["name"] for m in ps_resp.json().get("models", [])}
if default_model in loaded_names or f"{base}:latest" in loaded_names:
result["model"] = "loaded"
except Exception:
logger.debug("Ollama status check failed", exc_info=True)
return jsonify(result)
src/fabledassistant/services/generation_task.py
+24
View File
@@ -29,6 +29,26 @@ _TOOL_CALL_MARKER = re.compile(r"^\s*\[TOOL_CALLS\]\s*", re.IGNORECASE)
DB_FLUSH_INTERVAL = 5.0 # seconds between partial DB flushes
# Human-readable labels for each tool, shown in the status indicator
_TOOL_LABELS: dict[str, str] = {
"create_task": "Creating task",
"create_note": "Creating note",
"update_note": "Updating note",
"list_tasks": "Searching tasks",
"search_notes": "Searching notes",
"create_event": "Creating calendar event",
"list_events": "Searching calendar",
"search_events": "Searching calendar",
"update_event": "Updating calendar event",
"delete_event": "Removing calendar event",
"list_calendars": "Listing calendars",
"create_todo": "Creating todo",
"list_todos": "Listing todos",
"update_todo": "Updating todo",
"complete_todo": "Completing todo",
"delete_todo": "Removing todo",
}
async def _generate_title(messages: list[dict], model: str) -> str:
"""Ask the LLM for a concise conversation title."""
@@ -121,6 +141,7 @@ async def run_generation(
m for m in messages[1:-1]
if m.get("role") in ("user", "assistant") and m.get("content")
][-6:]
buf.append_event("status", {"status": "Analyzing your request..."})
intent = await classify_intent(user_content, tools, intent_model, history=intent_history)
if intent.should_execute:
logger.info(
@@ -133,6 +154,7 @@ async def run_generation(
intent.confidence, intent.tool_name,
)
if intent.should_execute:
buf.append_event("status", {"status": f"{_TOOL_LABELS.get(intent.tool_name, 'Working')}..."})
result = await execute_tool(user_id, intent.tool_name, intent.arguments)
logger.info("Intent-routed tool %s result: success=%s", intent.tool_name, result.get("success"))
@@ -159,6 +181,7 @@ async def run_generation(
})
continue # Next round: LLM streams response incorporating result
buf.append_event("status", {"status": "Generating response..." if _round == 0 else "Composing response..."})
async for chunk in stream_chat_with_tools(messages, model, tools=tools):
if buf.cancel_event.is_set():
cancelled = True
@@ -188,6 +211,7 @@ async def run_generation(
tool_name = fn.get("name", "")
arguments = fn.get("arguments", {})
logger.info("Executing tool: %s(%s)", tool_name, json.dumps(arguments)[:200])
buf.append_event("status", {"status": f"{_TOOL_LABELS.get(tool_name, 'Working')}..."})
result = await execute_tool(user_id, tool_name, arguments)
logger.info("Tool %s result: success=%s", tool_name, result.get("success"))
summary.md
+19 -1
View File
@@ -12,7 +12,7 @@
> Include file-level details in the commit body when the change is non-trivial.
## Last Updated
2026-02-18 — Phase 10 cont.: update_note task fields, list_tasks tool, update_todo CalDAV tool
2026-02-18 — Phase 11: streaming status transparency UX + model load state indicator
## Project Overview
Fabled Assistant is a self-hosted note-taking and task-tracking application with
@@ -550,6 +550,15 @@ When adding a new migration, follow these conventions:
- Per-conversation model selection via ModelSelector dropdown in chat header (persisted via PATCH)
- Dashboard inline chat input: model selector + note picker + textarea; creates conversation and navigates
- Model warming: default model pre-loaded into Ollama on dashboard mount via fire-and-forget POST
- **Model load state indicator:** `GET /api/chat/status` now queries `/api/tags` and `/api/ps` in
parallel. Model status has three values: `"not_found"` (not installed), `"cold"` (installed but
not in VRAM — first response will be slow), `"loaded"` (hot in VRAM, fast response).
`chatReady` is true for both `"cold"` and `"loaded"` since cold models still work.
AppHeader shows five distinct visual states: gray pulse (checking), red (Ollama down), orange
(model not installed), yellow pulse (cold), green (loaded). Each state has a short inline text
label ("Cold", "Ready", "Offline", etc.) visible without hovering, plus a tooltip with detail.
After `warmModel()` is called, a fast 5s polling loop runs for up to 60s until the indicator
transitions to green (instead of waiting for the 30s poll cycle).
- Hot/cold model indicators: `/api/chat/ps` proxies Ollama `/api/ps`; ModelSelector shows green/red emoji circles
- Ollama configured with `OLLAMA_MAX_LOADED_MODELS=2` and `OLLAMA_KEEP_ALIVE=30m`
- Timeout tuning: connect timeout 30s (cold model loading), warm timeout 300s, pull timeout 1800s
@@ -570,6 +579,15 @@ When adding a new migration, follow these conventions:
- `search_notes` — keyword search across notes and tasks
- Full CalDAV suite: `create_event`, `list_events`, `search_events`, `update_event`, `delete_event`,
`list_calendars`, `create_todo`, `list_todos`, `update_todo`, `complete_todo`, `delete_todo`
- **Streaming status transparency:** The backend emits `status` SSE events at each pipeline stage
so the user always sees what's happening instead of a blank progress dot. Stages:
(1) `"Analyzing your request..."` before intent classification; (2) human-readable tool label
(e.g. `"Creating calendar event..."`) before each tool executes (both intent-routed and native);
(3) `"Generating response..."` / `"Composing response..."` before each LLM streaming round.
Frontend: `chat.ts` stores `streamingStatus` ref, cleared on first content chunk or on done/error.
`ChatView.vue` shows a pulsing dot + italic label above the content while status is active, then
falls back to the blinking cursor when content streams in. `HomeView.vue` dashboard panel shows
the status label in place of `...` before any content arrives.
- **Intent routing:** Before streaming, a fast non-streaming LLM call classifies user intent and
extracts tool parameters (`services/intent.py`). If a tool call is detected, it executes directly
— bypassing the model's native (sometimes unreliable) tool calling API. Falls through to normal