Add _process_note() — a second LLM pass using the main model that
transforms raw capture text into a well-formed note with a genuine
summary title and formatted body. Replaces the previous behaviour of
using the captured text verbatim as both title and body.
The processing prompt instructs the model to:
- Generate a 3-8 word summary title (never a verbatim copy)
- Format the body appropriately: bullet lists for items, clean prose
for stream-of-thought, organised paragraphs for raw notes/fragments
- Preserve all original information without inventing new facts
The enrichment pass runs for both the intent-classified create_note
path and the fallback path. On LLM/parse failure it degrades safely
to the old verbatim behaviour.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- stream_chat and stream_chat_with_tools: remove read=300s per-chunk
timeout, replace with read=None. In httpx streaming mode, the read
timeout applies per-chunk — if Ollama pauses >300s while processing
a large input context before the first token, it raises ReadTimeout,
killing generation and leaving the assistant message as an empty stub.
With read=None the stream is unbounded; connect=30s still guards the
initial connection.
- chat_status_route: increase Ollama status check timeout 5s → 10s.
When Ollama is busy processing a large prompt it can be slow to
respond to /api/tags, causing the status indicator to briefly flip to
"offline" even though generation is running normally.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Remove all 6 CalDAV todo tools (create/list/update/complete/delete/search_todos)
from tools.py definitions, imports, execute_tool branches, intent routing rules,
generation_task labels/actions, and llm.py system prompt hints. CalDAV event
tools remain. Todo functions still exist in caldav.py but are no longer exposed.
- Quick-capture now uses a dedicated classify_capture_intent() with a focused
_CAPTURE_SYSTEM_PROMPT that always routes to a tool (never null). Tool set
expanded: create_note/task/event + update_note + research_topic.
- research_topic in quick-capture calls run_research_pipeline() directly (no SSE
buffer). run_research_pipeline() now accepts buf=None; all buf.append_event
calls are guarded so status events are skipped when no buffer is provided.
- Fallback note now always sets body=text (was empty for texts ≤80 chars).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Quart's send_file uses cache_timeout= not max_age=. The TypeError on
every /api/images/<id> request caused a 500, which the browser rendered
as alt text.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- tools.py: search_images result now includes 'embed' (ready-to-use
markdown image syntax) and 'citation' fields instead of raw 'local_url';
adds 'instructions' field so the model knows to render them verbatim
- llm.py: system prompt now explicitly tells the model to embed images
using the 'embed' field rather than describing or listing URLs
- markdown.ts: explicitly allow src/alt in PURIFY_OPTS_FULL so img tags
are never stripped by DOMPurify
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Images found via SearXNG are fetched server-side, stored on disk, and
served from /api/images/<id> — the user's browser never contacts the
original image host. Original URLs are preserved for citation.
New files:
- alembic/versions/0016_add_image_cache.py — image_cache table
- src/fabledassistant/models/image_cache.py — SQLAlchemy model
- src/fabledassistant/services/images.py — fetch/store/serve logic
- src/fabledassistant/routes/images.py — GET /api/images/<id>
Modified:
- config.py: IMAGE_CACHE_DIR (/data/images), IMAGE_MAX_BYTES (5 MB)
- research.py: _search_searxng_images() — SearXNG categories=images
- tools.py: _IMAGE_TOOLS def + search_images branch in execute_tool
- intent.py: search_images routing rule (explicit visual language only)
- app.py: register images_bp
- docker-compose.yml: image_cache named volume mounted at /data/images
- ToolCallCard.vue: "image_search" label
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Raise similarity threshold 0.30 → 0.45: only genuinely relevant notes
shown; loosely-related notes no longer pad the sidebar
- Increase max suggested notes 3 → 8 (zero added compute — threshold is
the real gate; the embedding call is fixed regardless of limit)
- semantic_search_notes now returns list[tuple[float, Note]] instead of
list[Note] so scores propagate through context_meta to the frontend
- Keyword fallback notes carry score=null (no cosine similarity available)
- ChatView sidebar shows % badge on each suggested note:
green ≥75%, amber 60–74%, muted <60%
Hovering reveals the raw score in a tooltip
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Single POST that classifies natural-language text and creates the
appropriate item (note, task, event, or todo) in one synchronous
request — no SSE, no conversation context needed.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The same empty-string model guard added to chat.py was missing from the
notes assist route. If default_model was stored as "" in the DB, the
assist route would pass "" to Ollama which responds with 400, surfaced
to the user as a "400" error in the assist panel.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The model occasionally passes title as a list (e.g. when asked to create
multiple notes at once), causing asyncpg DataError on the INSERT. Return a
clear error result so the model sees the problem and retries with individual
calls instead of crashing with an unhandled exception.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Add a Python fast-path regex (_PRIOR_WORK_REFS) in classify_intent that
detects phrases like "research you did", "note you made", "using your
research", "based on the research" etc. and returns no-tool immediately —
saving the 19s intent LLM call and correctly letting the main model answer
using search_notes/context rather than firing off a web search.
Also tighten the intent prompt rules for search_web: explicitly prohibit
using it for creative/brainstorming requests or when the user references
existing notes, and add a rule that creative/ideation questions ("think of",
"come up with", "brainstorm") always route to null (chat).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
When a user selected the 'Default' option in Settings, the dropdown
sent an empty string "" to the backend. The route saved it as a DB row,
which caused get_setting() to return "" instead of falling back to
Config defaults. The chat status endpoint then tried to match "" against
installed model names — always failing — resulting in model: "not_found"
and a permanently failing readiness indicator.
services/settings.py:
- Add delete_setting() helper: removes a setting row so get_setting()
correctly falls back to its hardcoded default argument
routes/settings.py:
- Import delete_setting
- When default_model or intent_model are saved as empty string, delete
the DB row instead of storing "" — cleanly restores Config fallback
routes/chat.py:
- chat_status_route: add explicit `or Config.OLLAMA_MODEL` guard for
any existing "" rows written before this fix (migration safety net)
- send_message and summarize routes: same guard on model resolution
so empty settings never cause silent generation failures
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
config.py:
- Default OLLAMA_INTENT_MODEL: qwen2.5:1.5b → qwen2.5:7b
- Startup will auto-pull and warm the new model on next container restart
intent.py:
- Replaced phrase-matching examples in search_web and research_topic rules
with semantic descriptions. The 7B model doesn't need example phrases to
understand intent — it can reason from the tool's purpose. Removes implied
usage patterns that caused misclassifications on conversational phrasing
(e.g. "I've been thinking about buying shirts, can you research this?").
- research_topic rule now explicitly covers any subject regardless of phrasing,
including shopping decisions, comparisons, how-things-work questions, etc.
- search_web rule clarified as "short summary, no note" vs research_topic's
"comprehensive written reference"
The 1.5B model required prescriptive phrase examples to route correctly; the
7B model has sufficient language understanding to classify from semantic intent.
Expected improvement: ~1-2s intent calls (vs 0.4-9s for the 1.5B model which
sometimes timed out or misclassified longer/conversational messages).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Two fixes for the intent model failing to route 'Research: X' messages
to research_topic:
1. Fast-path in classify_intent: if the message matches ^Research:\s+.+
(the exact format the UI Research button always sends), skip the LLM
call entirely and return research_topic with high confidence. This is
100% reliable and saves an unnecessary model call for this pattern.
2. Expanded research_topic rule examples in the system prompt to include
"Research: X" prefix format, shopping-style queries ("research where
to buy X"), and clarification that the topic is everything after the
keyword — improves LLM routing for natural-language research requests
that don't match the previous narrow examples.
Root cause: qwen2.5:1.5b misclassified "Research: where to buy three-
quarter sleeve tee shirts" as general chat (shopping query phrasing
combined with the colon confused the small model).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
When the intent model doesn't classify a research request (low confidence,
long message, etc.), the main model (qwen3) would correctly identify
research_topic itself and call it via the streaming tool loop. But
execute_tool("research_topic") only returns a dummy research_pending
placeholder, causing the model to see the result and retry — looping
up to MAX_TOOL_ROUNDS times.
Fix: filter research_topic out of stream_tools (the tool list given to
the main model via stream_chat_with_tools). research_topic is an
intent-only routing tool; the main model should never call it directly.
The full tools list (including research_topic) is still passed to
classify_intent so intent routing continues to work.
The _INTENT_ONLY_TOOLS frozenset makes this pattern explicit and
extensible for future intent-only tools.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
research.py:
- Parallelize all 5 SearXNG queries concurrently (200ms stagger via asyncio.gather)
- Parallelize all URL fetches in parallel (asyncio.gather) — up to 15 URLs at once
instead of sequential fetches; biggest performance win (was O(n) × 15s, now ~15s flat)
- _synthesize_note accepts buf: when provided uses stream_chat (num_ctx=16384,
num_predict=8192) to emit tokens into the chat buffer in real time so users see
the note being written; falls back to generate_completion when buf=None
- Added \n\n---\n\n separator before "Research complete!" to cleanly mark boundary
after streamed synthesis content
intent.py:
- classify_intent passes num_ctx=4096 to generate_completion — reduces VRAM pressure
and prefill time for the intent model call on every single request
generation_task.py:
- _INTENT_TRIGGER_WORDS frozenset (~50 action/object/date words) + _should_skip_intent()
skips intent classification for short messages (≤10 words) with no trigger words;
saves 400-800ms model call for conversational replies ("thanks", "okay", etc.)
- Added \n\n---\n\n separator before research "done" text in research_topic branch
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Ollama streams message.thinking tokens alongside message.content when
think=True — previously silently dropped. Now forwarded end-to-end.
Backend:
- llm.py: ChatChunk type gains "thinking" variant; stream_chat_with_tools
yields ChatChunk(type="thinking") for msg.thinking chunks before content
- generation_task.py: thinking chunks emit "thinking_chunk" SSE events
(not added to content_so_far — not persisted to DB)
Frontend:
- types/chat.ts: Message.thinking?: string (session-only, not from DB)
- stores/chat.ts: streamingThinking ref; thinking_chunk handler accumulates
chunks; on done, thinking carried into committed Message object then cleared
- ChatMessage.vue: collapsible <details class="thinking-block"> shown for
messages that have .thinking content (collapsed by default)
- ChatView.vue + ChatPanel.vue: live thinking block in streaming bubble —
open while only thinking is flowing, auto-collapses when content arrives;
typing indicator hidden while thinking is active
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Instead of relying solely on retry-on-500, poll /api/ps before starting
any LLM stream so the main model has time to fully load into VRAM.
- llm.py: add wait_for_model_loaded(model, timeout=90s) — polls /api/ps
every 2s, returns True when model appears in loaded list
- generation_task.py: launch model_load_task in parallel with build_context
and classify_intent (both use fast/small-model ops that don't need the
main model); after context is built, await the load task — shows
"Loading model..." status only if the user actually has to wait;
logs a warning and proceeds if 90s timeout elapses
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Tags are now a first-class field rather than being auto-extracted from
the note body. A new TagInput.vue chip component handles tag entry in
both editor views with autocomplete, Enter/comma/backspace UX, and
space-to-hyphen sanitization.
Backend:
- routes/notes.py: create reads tags from JSON; update accepts explicit
tags (omit = keep existing); append_tag writes to tags array with
dedup; suggest-tags accepts current_tags filter; remove extract_tags
- routes/tasks.py: same — explicit tags on create/update; remove extract_tags
- services/tag_suggestions.py: current_tags param replaces body extraction
- services/tools.py: create_note tool schema adds tags param; executor passes it
- services/llm.py: system prompt tells LLM to use tags param, not embed #tag in body
Frontend:
- components/TagInput.vue: new chip-based tag input (autocomplete, keyboard UX)
- NoteEditorView.vue / TaskEditorView.vue: tags ref loaded from note.tags;
TagInput placed between title and body; save/autosave include tags; suggest
now adds chips; fetchTagSuggestions passes current_tags; dirty tracks tags
- TiptapEditor.vue: remove fetchTags prop and TagSuggestion extension;
keep TagDecoration for legacy inline #tag highlighting
No DB migration needed — tags column already correct.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Tags with spaces (e.g. #science fiction) were breaking extraction because
TAG_RE only matched word characters — it would stop at the space and extract
#science instead of #science-fiction.
- TAG_RE (backend + frontend): add hyphens to character class so #science-fiction
is recognized as a single tag: [\w][\w-]* per segment
- System prompt: instruct LLM to use hyphens in multi-word tags, never spaces
- tag_suggestions.py: update prompt example + sanitize output by replacing
spaces with hyphens as a safety net regardless of LLM output
- append-tag route: sanitize incoming tag (spaces → hyphens) before appending
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Adds _stream_with_retry() async generator (wraps stream_chat_with_tools
with up to 2 retries on Ollama 500, 3s/6s delay). Previously only the
optimistic round 0 _fill_queue had retry logic. Two paths were still
bare: the declined-write-tool fresh stream, and the round 1+ stream.
Round 1 500s occur when tag suggestions (fire-and-forget inside
execute_tool) race the follow-up stream to the same model. The retry
waits for tag suggestions to complete before succeeding.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
ensure_model only downloads a model if missing; it does not load it
into VRAM. The frontend warm call only covers the chat model (and only
after a user opens the dashboard). This left qwen2.5:1.5b (intent) cold,
causing simultaneous cold-load 500s when the first chat arrived.
Now both Config.OLLAMA_MODEL and Config.OLLAMA_INTENT_MODEL are warmed
at startup (after ensuring they're installed) via a fire-and-forget
/api/generate call with keep_alive=30m. The embedding model is still
pulled but not warmed (it's loaded on demand during backfill).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
With optimistic streaming, intent (qwen2.5:1.5b) and the main stream
(qwen3:latest) start concurrently. When both models are cold-loading,
Ollama returns 500 for both simultaneously. The intent 500 was already
handled silently in classify_intent; the stream 500 now retries up to
2 times (3s then 6s delay) before propagating as an error. 500s only
occur on the first cold-load pair — subsequent requests hit warm models.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Start the main LLM stream immediately after build_context finishes instead
of waiting for intent classification to complete. Race the two concurrently:
- Intent wins before first token → cancel stream, execute tool (tool path
unchanged: confirmation, acknowledgment, multi-round loop all preserved)
- First token wins → discard intent, user sees output immediately
For pure chat messages (no tool needed, the common case) this eliminates
the full intent classification RTT from TTFT. For tool calls, intent
typically wins the race since it finishes before the main model produces
its first token, so tool behaviour is unchanged in practice.
Also extracts _drain_queue() as a module-level async generator helper.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Default OLLAMA_INTENT_MODEL to qwen2.5:1.5b in code instead of empty
- Add GET /api/settings/models endpoint returning installed models and defaults
- Validate intent_model against installed models on save (same as default_model)
- Replace intent model text input with a dropdown of installed models
- Add chat model dropdown to Assistant settings section
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Replaces the hardcoded num_ctx=32768 KV cache allocation with a
configurable env var defaulting to 8192. This significantly reduces
VRAM pressure when multiple services share the GPU.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
favicon.svg:
- Light mode: replace near-black fill (#2d3748) with indigo brand color
(#6366f1 fill, #4f46e5 stroke, #a5b4fc page lines) — distinctive and
high-contrast without the dark/black appearance
- Dark mode unchanged
email.py:
- Add _EMAIL_LOGO_SVG: inline SVG with white palette for rendering on
the indigo header (white book, lavender lines, gold sparkle)
- Add _email_html(title, body): shared template wrapper — gray outer
background, white card with border-radius, indigo header with logo +
app name, content area, footer
notifications.py:
- Import and use _email_html for all six email functions: security alert,
password reset, password reset success, invitation, task reminder,
test email
- Clean up all inline HTML to match the new card layout and spacing
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Without pool_pre_ping, stale pooled connections (left over after a Postgres
restart or network blip) cause immediate query failures. SQLAlchemy then
propagates the error rather than transparently reconnecting, which crashes
the Quart request handler and triggers a Swarm restart loop.
- pool_pre_ping=True: issues a lightweight SELECT 1 before each checkout;
discards and replaces stale connections silently
- pool_recycle=1800: recycles connections every 30 minutes to prevent
long-idle connections from going stale at the TCP/firewall level
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Backend security & correctness:
- Add rate_limit.py: sliding-window rate limiter (asyncio) applied to login,
register, forgot/reset password endpoints (10/60s or 5/300s per IP)
- app.py: add security headers in after_request (X-Frame-Options, CSP,
X-Content-Type-Options, Referrer-Policy) using setdefault to preserve SSE headers
- auth.py: refactor duplicate login_required/admin_required into shared _check_auth()
- config.py: add TRUST_PROXY_HEADERS for proxy-aware client IP resolution
- routes/auth.py: rate limiting, _client_ip() helper, cleaned-up reset_password route
- routes/chat.py, notes.py, tasks.py: int() DoS fix on last_event_id; limit capped
at 500; date.fromisoformat() wrapped in try/except → 400 on invalid dates
- services/auth.py: fix Setting.user_id update bug (filter on NULL not user.id);
reset_password_with_token returns int|None (user_id) instead of bool
- services/backup.py: add _security_notice to full backup JSON export
- services/assist.py: system prompt explicitly preserves markdown list structure
and nested indented sub-items
Infrastructure:
- docker-compose.yml: add healthcheck on app service (/api/health, 10s interval)
- .dockerignore: prevent secrets/node_modules/__pycache__/.env.* leaking into build
Frontend bug fixes:
- TaskCard.vue, TaskViewerView.vue: fix isOverdue() timezone bug (ISO string compare)
- useAssist.ts: accept() now resets state to idle when document changed since proposal
- stores/chat.ts: fix memory leak in _pollUntilLoaded() (try/catch around fetchStatus)
- TiptapEditor.vue: selection offset uses closest-match strategy (not first-match)
- utils/markdown.ts: explicit DOMPurify config with FORCE_BODY; remove as const
(DOMPurify expects mutable string[])
New features:
- Auto-save (5-minute interval) in NoteEditorView and TaskEditorView — only when
editing an existing dirty record; silent on error, shows "Auto-saved" toast
- sectionParser.ts: top-level bullet/numbered list items are now individual sections
in the AI Assist panel (previously treated as one undifferentiated block)
- editor-shared.css: extracted ~500 lines of CSS duplicated between both editors;
includes .inline-assist-btn at global scope (required for teleported elements)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Context sidebar + note title:
- ChatView: replace ephemeral context pills with a persistent right-panel sidebar;
auto-found notes accumulate across turns; attached note shows with pin icon;
× button excludes a note from future auto-search; hidden on mobile
- routes/chat.py: batch-fetch note titles via get_notes_by_ids() and inject
context_note_title into each message dict at conversation load time
- notes.py: add get_notes_by_ids() batch fetch helper
- types/chat.ts: add context_note_title field to Message interface
- stores/chat.ts: sendMessage accepts optional 5th arg contextNoteTitle,
included in optimistic user message
- ChatMessage.vue: context badge shows note title instead of 'Note #N'
Expanded LLM tool suite (all with intent router rules + ToolCallCard display):
- delete_note / delete_task: permanent delete with user confirmation (write tool),
type-safe (refuse to delete wrong type), clears note context cache on success
- get_note: fetch full note body by query (search_notes returns only 200-char preview)
- list_notes: browse notes by recency/keyword/tags with limit; notes only
- update_note: add tags + tag_mode (replace/add/remove) parameters
- search_notes: add optional type filter ("note" | "task")
- search_todos (CalDAV): keyword-filter todos, companion to list_todos
- caldav.py: add search_todos() built on top of list_todos()
- generation_task.py: register new tools in _WRITE_TOOLS, _TOOL_LABELS, _TOOL_ACTIONS
- llm.py: update available actions list and guidance in system prompt
- intent.py: routing rules for all new tools
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- New NoteEmbedding model + migration 0014 stores float embeddings (JSONB)
- services/embeddings.py: get_embedding, upsert_note_embedding,
semantic_search_notes (cosine similarity), backfill_note_embeddings
- build_context() now tries semantic search first, falls back to keyword search;
accepts cached_note_ids to reuse last-turn notes and stabilise the system
prompt prefix for Ollama's KV cache
- generation_buffer.py: per-conversation note ID cache (get/set/clear)
- generation_task.py: passes cached IDs into build_context, updates cache
after each turn, and invalidates it after create_note/update_note/create_task
- app.py: pulls nomic-embed-text at startup and launches a background backfill
to embed all existing notes (30 s delay so Ollama has time to load the model)
- routes/notes.py + services/tools.py: fire-and-forget embedding update on
every note create or update via the API or LLM tool calls
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
When a conversation exceeds 20 messages (10 exchanges), the oldest
messages are summarized into a compact 3-5 sentence paragraph using the
intent model, and only the most recent 6 messages are passed verbatim.
The summary is injected into the system prompt so the model retains
context without the full token cost. For short conversations the check
is O(1) and returns immediately. The status indicator shows
"Summarizing conversation history..." when the LLM call is needed.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
stream_chat_with_tools now accepts a think parameter. run_generation
forwards it to Ollama. The message POST route reads think from the
request body. ChatView passes think=true so qwen3 uses chain-of-thought
reasoning for full conversations; the dashboard widget and ChatPanel
omit it, staying fast. Dashboard button updated to "Think it through
in Chat →" to signal the deeper capability.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Before executing any write tool (create/update/delete), the backend now
pauses with an asyncio.Future and emits a tool_pending SSE event. The
frontend displays a ToolConfirmCard with Accept and Decline buttons.
Clicking Accept resolves the Future and proceeds; Decline records a
declined tool_call chip and falls through to regular streaming. Typing
single-word yes/no responses (e.g. "yes", "cancel") also works as
confirmation. 120s timeout auto-declines if no response.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
When the intent router detects a tool call, the acknowledgment sentence
and the tool now execute concurrently via asyncio.gather. The acknowledgment
uses the small intent model (already in VRAM) with max_tokens=40, so it
completes in ~200-400ms — the user sees text almost immediately instead of
staring at a status label for the full main-model TTFT (~22s).
The acknowledgment text is:
- Streamed to the client as a chunk event (clears the status spinner)
- Included in the assistant message for round 1 so the main LLM continues
coherently from where the acknowledgment left off
- Recorded in TTFT timing (acknowledgment counts as first token)
Varied phrasing is enforced in the system prompt so responses feel natural
rather than formulaic.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Pinned note: full body restored (truncation is wrong when the user is
explicitly asking about that note's content).
Auto-notes: restored to 2000 chars (800 was too restrictive for useful context).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Auto-notes (keyword-matched): 2000 → 800 chars each (×3 max = 6000 → 2400 chars).
Pinned note (explicit context): was unbounded → capped at 4000 chars with [truncated] marker.
The main post-GPU bottleneck is TTFT caused by the prefill phase — the model
processing the full input before generating any tokens. Shorter context =
faster prefill. Users can ask follow-up questions for more detail.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
If OLLAMA_INTENT_MODEL is configured and differs from OLLAMA_MODEL,
both are pulled concurrently as fire-and-forget tasks at startup.
Deduplicates via a set so pulling the same model twice is never attempted.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Docker Compose:
- Enable Ollama GPU passthrough (nvidia, count: all) in both dev and prod files
- Add OLLAMA_FLASH_ATTENTION=1 (faster attention on GPU in both files)
- Add OLLAMA_MAX_LOADED_MODELS=2 and OLLAMA_KEEP_ALIVE=30m to prod (was already in dev)
- Remove 8G memory limit from prod Ollama service (CPU-bound constraint, no longer valid)
llm.py:
- Increase num_ctx 16384 → 32768 in stream_chat and stream_chat_with_tools (GPU VRAM allows it)
- Increase num_predict cap 4096 → 8192 for tool-augmented responses
generation_task.py:
- Parallelize build_context, get_tools_for_user, and get_setting all from the start
- As soon as tools list is ready (fast DB call), launch classify_intent as an asyncio.Task
- Await build_context and classify_intent together via asyncio.gather
- Intent result is pre-computed before the generation loop; loop just reads pre_intent on round 0
- intent_ms timing now reflects wall-clock time from intent start to completion
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- build_context() moved from route handler into run_generation() background task.
The 202 response now returns immediately; client connects to SSE before
note search / URL fetch begins, so 'Building context...' status is visible.
- _generate_title() runs in a fire-and-forget asyncio.create_task() after the
'done' SSE event fires. Users see their response complete 2–5s sooner on new
conversations; title appears later in the sidebar without blocking the stream.
- generate_completion() now sets think:False and accepts a max_tokens limit.
Intent classifier passes max_tokens=200 (JSON only), title generator passes
max_tokens=30 (short title), eliminating qwen3 thinking-mode overhead on these
auxiliary calls.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Bug fix:
- ChatView.vue onMounted now skips fetchConversation when the conversation
is already loaded in the store (same guard that the convId watcher uses).
This prevents duplicate assistant messages when navigating from the
dashboard inline chat to /chat/:id after streaming completes.
Generation timing:
- logging.py: add log_generation() — persists per-generation timing
breakdown to app_logs (category=usage, action=generation) including
model, total_ms, intent_ms, ttft_ms, generation_ms, and per-tool timings.
Queryable via existing admin log viewer.
- generation_task.py: collect wall-clock timestamps at every pipeline stage:
intent classification, per-tool execution (both intent-routed and native),
time-to-first-token (measured from generation start to first content chunk),
LLM streaming round duration. Logs via log_generation() and includes timing
in the SSE 'done' event payload.
- types/chat.ts: add GenerationTiming interface; add optional timing field
to Message.
- chat.ts: capture timing from done event and attach to assistant message.
- ChatMessage.vue: show timing footer on assistant messages with breakdown:
"⏱ 4.2s total · first token 0.8s · analyzed 0.3s · created event 0.4s
· generated 3.5s". Visible this session; persisted to app_logs for
cross-session benchmarking.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Streaming status transparency:
- generation_task.py emits 'status' SSE events at each pipeline stage:
"Analyzing your request..." before intent classification, tool label
before each tool execution, "Generating/Composing response..." before
each LLM streaming round
- chat.ts adds streamingStatus ref; cleared on first chunk or done/error;
includes fast 5s poll loop after warmModel() until model shows as loaded
- ChatView.vue shows pulsing dot + italic status label above content area;
falls back to blinking cursor once content arrives
- HomeView.vue shows status label in dashboard panel instead of '...'
Model load state indicator:
- /api/chat/status now queries /api/tags and /api/ps in parallel to
distinguish installed-but-cold vs loaded-in-VRAM model states
- New model status values: 'not_found' | 'cold' | 'loaded' (was 'ready')
- chatReady true for both 'cold' and 'loaded' (cold models still work)
- AppHeader shows 5 states: gray pulse (checking), red (Ollama down),
orange (not installed), yellow pulse (cold), green (loaded)
- Inline short label ("Cold", "Ready", "Offline", etc.) visible without
hovering; detailed tooltip on hover
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- update_note: extend with status/priority/due_date fields so task attributes
can be changed via chat (mark done, set priority, move due date). body is now
optional — task field updates work without touching content.
- list_tasks: new core tool with status/priority/due_before/due_after/limit
filters backed by list_notes(is_task=True). Enables queries like
"overdue tasks", "high priority tasks", "what's in progress".
- update_todo: new CalDAV tool to modify VTODO summary, due date, description,
and priority — follows update_event pattern (modify component, rebuild ical,
save). Completes the CalDAV todo CRUD suite.
- tools.py: add update_todo import + execute case (type: todo_updated)
- llm.py: add list_tasks and update_todo to available actions + guidance
- intent.py: routing rules for mark-done/priority/due-date → update_note,
overdue/in-progress/high-priority queries → list_tasks, CalDAV todo updates
→ update_todo
- ToolCallCard.vue: tasks list block (linked titles + due + priority badges),
todo_updated label, tool-task-priority CSS classes
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Mistral didn't reliably use Ollama's structured tool calling API — it wrote
tool calls as JSON text instead of invoking them. This adds an intent routing
layer that classifies user intent via a fast non-streaming LLM call before
streaming, executing detected tools directly and bypassing native tool calling.
- Change default OLLAMA_MODEL from mistral to qwen3
- Add intent.py: classify_intent() with JSON parsing and fallback regex
- Integrate intent routing into generation_task.py round 0
- Add all-day event support (iCalendar DATE values) to CalDAV service
- Add recurring event support (RRULE) to CalDAV service and tool definition
- Improve create_event tool description for descriptive titles
- Enhance system prompt with structured tool usage guidance
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- CalDAV integration: per-user calendar config, create/list/search events
via caldav library, LLM tools for calendar operations from chat
- LLM-suggested tags: new tag_suggestions service prompts LLM with existing
tags and note content to suggest 3-5 relevant tags; exposed via API
endpoints (suggest-tags, append-tag); integrated into editor views
(suggest button + clickable pills) and chat tool calls (pills in
ToolCallCard with one-click apply)
- Settings/model UI refinements, generation task improvements
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
bvandeusen