Add streaming status UX and model load state indicator

Streaming status transparency:
- generation_task.py emits 'status' SSE events at each pipeline stage:
  "Analyzing your request..." before intent classification, tool label
  before each tool execution, "Generating/Composing response..." before
  each LLM streaming round
- chat.ts adds streamingStatus ref; cleared on first chunk or done/error;
  includes fast 5s poll loop after warmModel() until model shows as loaded
- ChatView.vue shows pulsing dot + italic status label above content area;
  falls back to blinking cursor once content arrives
- HomeView.vue shows status label in dashboard panel instead of '...'

Model load state indicator:
- /api/chat/status now queries /api/tags and /api/ps in parallel to
  distinguish installed-but-cold vs loaded-in-VRAM model states
- New model status values: 'not_found' | 'cold' | 'loaded' (was 'ready')
- chatReady true for both 'cold' and 'loaded' (cold models still work)
- AppHeader shows 5 states: gray pulse (checking), red (Ollama down),
  orange (not installed), yellow pulse (cold), green (loaded)
- Inline short label ("Cold", "Ready", "Offline", etc.) visible without
  hovering; detailed tooltip on hover

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
2026-02-18 17:54:37 -05:00
parent b8acd05ec4
commit fbce540638
8 changed files with 171 additions and 24 deletions
+19 -1
View File
@@ -12,7 +12,7 @@
> Include file-level details in the commit body when the change is non-trivial.
## Last Updated
2026-02-18 — Phase 10 cont.: update_note task fields, list_tasks tool, update_todo CalDAV tool
2026-02-18 — Phase 11: streaming status transparency UX + model load state indicator
## Project Overview
Fabled Assistant is a self-hosted note-taking and task-tracking application with
@@ -550,6 +550,15 @@ When adding a new migration, follow these conventions:
- Per-conversation model selection via ModelSelector dropdown in chat header (persisted via PATCH)
- Dashboard inline chat input: model selector + note picker + textarea; creates conversation and navigates
- Model warming: default model pre-loaded into Ollama on dashboard mount via fire-and-forget POST
- **Model load state indicator:** `GET /api/chat/status` now queries `/api/tags` and `/api/ps` in
parallel. Model status has three values: `"not_found"` (not installed), `"cold"` (installed but
not in VRAM — first response will be slow), `"loaded"` (hot in VRAM, fast response).
`chatReady` is true for both `"cold"` and `"loaded"` since cold models still work.
AppHeader shows five distinct visual states: gray pulse (checking), red (Ollama down), orange
(model not installed), yellow pulse (cold), green (loaded). Each state has a short inline text
label ("Cold", "Ready", "Offline", etc.) visible without hovering, plus a tooltip with detail.
After `warmModel()` is called, a fast 5s polling loop runs for up to 60s until the indicator
transitions to green (instead of waiting for the 30s poll cycle).
- Hot/cold model indicators: `/api/chat/ps` proxies Ollama `/api/ps`; ModelSelector shows green/red emoji circles
- Ollama configured with `OLLAMA_MAX_LOADED_MODELS=2` and `OLLAMA_KEEP_ALIVE=30m`
- Timeout tuning: connect timeout 30s (cold model loading), warm timeout 300s, pull timeout 1800s
@@ -570,6 +579,15 @@ When adding a new migration, follow these conventions:
- `search_notes` — keyword search across notes and tasks
- Full CalDAV suite: `create_event`, `list_events`, `search_events`, `update_event`, `delete_event`,
`list_calendars`, `create_todo`, `list_todos`, `update_todo`, `complete_todo`, `delete_todo`
- **Streaming status transparency:** The backend emits `status` SSE events at each pipeline stage
so the user always sees what's happening instead of a blank progress dot. Stages:
(1) `"Analyzing your request..."` before intent classification; (2) human-readable tool label
(e.g. `"Creating calendar event..."`) before each tool executes (both intent-routed and native);
(3) `"Generating response..."` / `"Composing response..."` before each LLM streaming round.
Frontend: `chat.ts` stores `streamingStatus` ref, cleared on first content chunk or on done/error.
`ChatView.vue` shows a pulsing dot + italic label above the content while status is active, then
falls back to the blinking cursor when content streams in. `HomeView.vue` dashboard panel shows
the status label in place of `...` before any content arrives.
- **Intent routing:** Before streaming, a fast non-streaming LLM call classifies user intent and
extracts tool parameters (`services/intent.py`). If a tool call is detected, it executes directly
— bypassing the model's native (sometimes unreliable) tool calling API. Falls through to normal