Add LLM tool calling for creating tasks, notes, and searching from chat
Ollama tool/function calling integration allows the LLM to create tasks, create notes, and search existing notes on behalf of the user during chat. Multi-round tool loop (max 5 rounds) lets the model execute tools then produce a natural language response. Tool results are persisted in a new JSONB column on messages and rendered as compact cards with linked titles. - Migration 0013: add tool_calls JSONB column to messages - New services/tools.py: tool definitions + execute_tool dispatcher - llm.py: ChatChunk dataclass, stream_chat_with_tools(), date in system prompt - generation_task.py: multi-round tool call loop with SSE tool_call events - Frontend: ToolCallRecord type, streamingToolCalls in store, ToolCallCard component, rendering in ChatMessage and ChatView streaming bubble Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
+21
-10
@@ -12,7 +12,7 @@
|
||||
> Include file-level details in the commit body when the change is non-trivial.
|
||||
|
||||
## Last Updated
|
||||
2026-02-14 — Phase 6.1: Model status colors, timeout tuning, SSE error feedback
|
||||
2026-02-14 — Phase 7: LLM tool calling (create tasks, create notes, search notes from chat)
|
||||
|
||||
## Project Overview
|
||||
Fabled Assistant is a self-hosted note-taking and task-tracking application with
|
||||
@@ -202,10 +202,11 @@ for AI-assisted features.
|
||||
### Messages
|
||||
- `id` (int PK), `conversation_id` (FK to conversations, CASCADE), `role` (str: system/user/assistant),
|
||||
`content` (str), `status` (str, default `'complete'` — `complete`/`generating`/`error`),
|
||||
`context_note_id` (nullable FK to notes, SET NULL), `created_at`
|
||||
`context_note_id` (nullable FK to notes, SET NULL), `tool_calls` (nullable JSONB), `created_at`
|
||||
- Index on `conversation_id`
|
||||
- `context_note_id` tracks which note was attached as context when message was sent
|
||||
- `to_dict()` returns: `id`, `conversation_id`, `role`, `content`, `status`, `context_note_id`, `created_at`
|
||||
- `tool_calls` stores an array of tool call records `[{function, arguments, result, status}]` for assistant messages that used tools
|
||||
- `to_dict()` returns: `id`, `conversation_id`, `role`, `content`, `status`, `context_note_id`, `tool_calls`, `created_at`
|
||||
|
||||
## Project Structure (Current)
|
||||
```
|
||||
@@ -230,7 +231,8 @@ fabledassistant/
|
||||
│ ├── 0009_add_message_status.py # Add status column to messages table (default 'complete')
|
||||
│ ├── 0010_add_app_logs_table.py # App logs table for audit, usage, and error logging
|
||||
│ ├── 0011_add_password_reset_tokens.py # Password reset tokens table
|
||||
│ └── 0012_add_invitation_tokens.py # Invitation tokens table
|
||||
│ ├── 0012_add_invitation_tokens.py # Invitation tokens table
|
||||
│ └── 0013_add_tool_calls_to_messages.py # Add tool_calls JSONB column to messages
|
||||
├── src/
|
||||
│ └── fabledassistant/
|
||||
│ ├── __init__.py
|
||||
@@ -258,10 +260,11 @@ fabledassistant/
|
||||
│ │ ├── auth.py # Auth business logic: hash_password, verify_password, create_user, authenticate, get_user_by_id, is_registration_open, list_users, delete_user, set_registration_open, password reset tokens, invitation tokens (create, validate, register_with_invitation, list_pending, revoke)
|
||||
│ │ ├── backup.py # Backup/restore: export_full_backup, export_user_backup, restore_full_backup
|
||||
│ │ ├── notes.py # CRUD with user_id isolation, is_task filter, convert, backlinks, search_notes_for_context
|
||||
│ │ ├── llm.py # Ollama interaction: build_context with user_id, streaming, URL fetching
|
||||
│ │ ├── llm.py # Ollama interaction: build_context with user_id, streaming (stream_chat + stream_chat_with_tools), ChatChunk dataclass, URL fetching
|
||||
│ │ ├── chat.py # Conversation CRUD with user_id isolation, add_message, save/summarize as note (LLM-titled, chat-tagged)
|
||||
│ │ ├── generation_buffer.py # In-memory SSE event buffer with cancel_event, reconnect support, auto-cleanup; supports chat (int keys) and assist (string keys)
|
||||
│ │ ├── generation_task.py # Background asyncio tasks: run_generation (chat, DB flush, titles) + run_assist_generation (lightweight, no DB)
|
||||
│ │ ├── generation_task.py # Background asyncio tasks: run_generation (chat, DB flush, titles, tool loop) + run_assist_generation (lightweight, no DB)
|
||||
│ │ ├── tools.py # LLM tool definitions (create_task, create_note, search_notes) + execute_tool dispatcher
|
||||
│ │ ├── settings.py # Settings CRUD with user_id isolation: get_setting, set_setting, set_settings_batch, get_all_settings
|
||||
│ │ ├── logging.py # App logging: log_audit, log_usage, log_error, get_logs, get_log_stats, delete_old_logs, start_log_retention_loop
|
||||
│ │ ├── email.py # SMTP email service: get_smtp_config, is_smtp_configured, send_email, send_test_email
|
||||
@@ -295,7 +298,7 @@ fabledassistant/
|
||||
│ ├── types/
|
||||
│ │ ├── auth.ts # User interface, AuthStatus interface
|
||||
│ │ ├── note.ts # Note interface (with status, priority, due_date, is_task) + TaskStatus, TaskPriority types + NoteListResponse
|
||||
│ │ ├── chat.ts # Message, Conversation, ConversationDetail, ContextMeta, OllamaModel, RunningModel, OllamaStatus interfaces
|
||||
│ │ ├── chat.ts # ToolCallRecord, Message, Conversation, ConversationDetail, ContextMeta, OllamaModel, RunningModel, OllamaStatus interfaces
|
||||
│ │ ├── settings.ts # AppSettings interface, ModelInfo interface (name, description, size, bestFor, category)
|
||||
│ │ └── task.ts # Task = re-export of Note; TaskListResponse
|
||||
│ ├── extensions/
|
||||
@@ -328,7 +331,8 @@ fabledassistant/
|
||||
│ │ ├── ChatPanel.vue # Slide-out chat panel (right side overlay, receives contextNoteId prop), bubble-style messages, floating dark input, note picker, context pills with promote/exclude
|
||||
│ │ ├── ModelSelector.vue # Model dropdown (v-model pattern): fetches installed + running models, hot/cold indicators
|
||||
│ │ ├── DashboardChatInput.vue # Inline chat bar: ModelSelector + note picker + textarea + send button; emits submit event
|
||||
│ │ ├── ChatMessage.vue # Message bubble: markdown rendering, configurable assistant name label, "Save as Note" action on assistant messages
|
||||
│ │ ├── ToolCallCard.vue # Compact card for tool call results (created task/note link, search results, errors)
|
||||
│ │ ├── ChatMessage.vue # Message bubble: markdown rendering, tool call cards, configurable assistant name label, "Save as Note" action on assistant messages
|
||||
│ │ ├── NoteCard.vue # Card with rendered markdown preview (v-html), TagPill, tag-click emit, hover edit button
|
||||
│ │ ├── TaskCard.vue # Card with rendered preview (body not description), StatusBadge (clickable), PriorityBadge, due date, tags, hover edit button
|
||||
│ │ ├── StatusBadge.vue # Color-coded status badge, optional clickable cycling
|
||||
@@ -422,7 +426,7 @@ container startup.
|
||||
|
||||
### Migration Chain
|
||||
```
|
||||
0001_create_notes_table.py → 0002_create_tasks_table.py → 0003_task_note_companion.py → 0004_merge_tasks_into_notes.py → 0005_add_chat_tables.py → 0006_add_settings_table.py → 0007_add_title_and_updated_at_indexes.py → 0008_add_users_and_user_id.py → 0009_add_message_status.py → 0010_add_app_logs_table.py → 0011_add_password_reset_tokens.py → 0012_add_invitation_tokens.py
|
||||
0001_create_notes_table.py → 0002_create_tasks_table.py → 0003_task_note_companion.py → 0004_merge_tasks_into_notes.py → 0005_add_chat_tables.py → 0006_add_settings_table.py → 0007_add_title_and_updated_at_indexes.py → 0008_add_users_and_user_id.py → 0009_add_message_status.py → 0010_add_app_logs_table.py → 0011_add_password_reset_tokens.py → 0012_add_invitation_tokens.py → 0013_add_tool_calls_to_messages.py
|
||||
```
|
||||
|
||||
### How Migrations Run
|
||||
@@ -531,6 +535,13 @@ When adding a new migration, follow these conventions:
|
||||
- Ollama configured with `OLLAMA_MAX_LOADED_MODELS=2` and `OLLAMA_KEEP_ALIVE=30m`
|
||||
- Timeout tuning: connect timeout 30s (cold model loading), warm timeout 300s, pull timeout 1800s
|
||||
- SSE reconnection failure shows error toast instead of silently recovering
|
||||
- **LLM tool calling:** Models with tool support can create tasks, create notes, and search notes
|
||||
on behalf of the user during chat. Multi-round tool loop (max 5 rounds) allows the LLM to
|
||||
execute tools and then produce a natural language response incorporating results. Tool call
|
||||
results are persisted in message `tool_calls` JSONB column and rendered as compact ToolCallCard
|
||||
components (linked titles for created items, search result lists). SSE emits `tool_call` events
|
||||
for real-time rendering during streaming. System prompt includes today's date for relative date
|
||||
resolution. Graceful degradation: models without tool support respond normally.
|
||||
|
||||
### Authentication & User Management
|
||||
- Session cookie auth with bcrypt, first-user-is-admin, orphaned data claiming
|
||||
@@ -571,7 +582,7 @@ When adding a new migration, follow these conventions:
|
||||
- Multi-stage Dockerfile: Node build → Python runtime, auto-migration on startup
|
||||
- Docker Compose (dev) + Docker Swarm production stack with secrets, overlay networks, health checks
|
||||
- Quart serves Vue SPA from static + REST API under `/api/`; 404 handler for SPA routing
|
||||
- Alembic migrations: 12 migrations, all raw SQL with idempotency guards (IF NOT EXISTS, DO $$ BEGIN...EXCEPTION)
|
||||
- Alembic migrations: 13 migrations, all raw SQL with idempotency guards (IF NOT EXISTS, DO $$ BEGIN...EXCEPTION)
|
||||
- Config from env vars + Docker secrets file support (`_read_secret`)
|
||||
|
||||
## Development Workflow
|
||||
|
||||
Reference in New Issue
Block a user