Add streaming status UX and model load state indicator
Streaming status transparency:
- generation_task.py emits 'status' SSE events at each pipeline stage:
"Analyzing your request..." before intent classification, tool label
before each tool execution, "Generating/Composing response..." before
each LLM streaming round
- chat.ts adds streamingStatus ref; cleared on first chunk or done/error;
includes fast 5s poll loop after warmModel() until model shows as loaded
- ChatView.vue shows pulsing dot + italic status label above content area;
falls back to blinking cursor once content arrives
- HomeView.vue shows status label in dashboard panel instead of '...'
Model load state indicator:
- /api/chat/status now queries /api/tags and /api/ps in parallel to
distinguish installed-but-cold vs loaded-in-VRAM model states
- New model status values: 'not_found' | 'cold' | 'loaded' (was 'ready')
- chatReady true for both 'cold' and 'loaded' (cold models still work)
- AppHeader shows 5 states: gray pulse (checking), red (Ollama down),
orange (not installed), yellow pulse (cold), green (loaded)
- Inline short label ("Cold", "Ready", "Offline", etc.) visible without
hovering; detailed tooltip on hover
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
+19
-1
@@ -12,7 +12,7 @@
|
||||
> Include file-level details in the commit body when the change is non-trivial.
|
||||
|
||||
## Last Updated
|
||||
2026-02-18 — Phase 10 cont.: update_note task fields, list_tasks tool, update_todo CalDAV tool
|
||||
2026-02-18 — Phase 11: streaming status transparency UX + model load state indicator
|
||||
|
||||
## Project Overview
|
||||
Fabled Assistant is a self-hosted note-taking and task-tracking application with
|
||||
@@ -550,6 +550,15 @@ When adding a new migration, follow these conventions:
|
||||
- Per-conversation model selection via ModelSelector dropdown in chat header (persisted via PATCH)
|
||||
- Dashboard inline chat input: model selector + note picker + textarea; creates conversation and navigates
|
||||
- Model warming: default model pre-loaded into Ollama on dashboard mount via fire-and-forget POST
|
||||
- **Model load state indicator:** `GET /api/chat/status` now queries `/api/tags` and `/api/ps` in
|
||||
parallel. Model status has three values: `"not_found"` (not installed), `"cold"` (installed but
|
||||
not in VRAM — first response will be slow), `"loaded"` (hot in VRAM, fast response).
|
||||
`chatReady` is true for both `"cold"` and `"loaded"` since cold models still work.
|
||||
AppHeader shows five distinct visual states: gray pulse (checking), red (Ollama down), orange
|
||||
(model not installed), yellow pulse (cold), green (loaded). Each state has a short inline text
|
||||
label ("Cold", "Ready", "Offline", etc.) visible without hovering, plus a tooltip with detail.
|
||||
After `warmModel()` is called, a fast 5s polling loop runs for up to 60s until the indicator
|
||||
transitions to green (instead of waiting for the 30s poll cycle).
|
||||
- Hot/cold model indicators: `/api/chat/ps` proxies Ollama `/api/ps`; ModelSelector shows green/red emoji circles
|
||||
- Ollama configured with `OLLAMA_MAX_LOADED_MODELS=2` and `OLLAMA_KEEP_ALIVE=30m`
|
||||
- Timeout tuning: connect timeout 30s (cold model loading), warm timeout 300s, pull timeout 1800s
|
||||
@@ -570,6 +579,15 @@ When adding a new migration, follow these conventions:
|
||||
- `search_notes` — keyword search across notes and tasks
|
||||
- Full CalDAV suite: `create_event`, `list_events`, `search_events`, `update_event`, `delete_event`,
|
||||
`list_calendars`, `create_todo`, `list_todos`, `update_todo`, `complete_todo`, `delete_todo`
|
||||
- **Streaming status transparency:** The backend emits `status` SSE events at each pipeline stage
|
||||
so the user always sees what's happening instead of a blank progress dot. Stages:
|
||||
(1) `"Analyzing your request..."` before intent classification; (2) human-readable tool label
|
||||
(e.g. `"Creating calendar event..."`) before each tool executes (both intent-routed and native);
|
||||
(3) `"Generating response..."` / `"Composing response..."` before each LLM streaming round.
|
||||
Frontend: `chat.ts` stores `streamingStatus` ref, cleared on first content chunk or on done/error.
|
||||
`ChatView.vue` shows a pulsing dot + italic label above the content while status is active, then
|
||||
falls back to the blinking cursor when content streams in. `HomeView.vue` dashboard panel shows
|
||||
the status label in place of `...` before any content arrives.
|
||||
- **Intent routing:** Before streaming, a fast non-streaming LLM call classifies user intent and
|
||||
extracts tool parameters (`services/intent.py`). If a tool call is detected, it executes directly
|
||||
— bypassing the model's native (sometimes unreliable) tool calling API. Falls through to normal
|
||||
|
||||
Reference in New Issue
Block a user