Add LLM tool calling for creating tasks, notes, and searching from chat
Ollama tool/function calling integration allows the LLM to create tasks, create notes, and search existing notes on behalf of the user during chat. Multi-round tool loop (max 5 rounds) lets the model execute tools then produce a natural language response. Tool results are persisted in a new JSONB column on messages and rendered as compact cards with linked titles. - Migration 0013: add tool_calls JSONB column to messages - New services/tools.py: tool definitions + execute_tool dispatcher - llm.py: ChatChunk dataclass, stream_chat_with_tools(), date in system prompt - generation_task.py: multi-round tool call loop with SSE tool_call events - Frontend: ToolCallRecord type, streamingToolCalls in store, ToolCallCard component, rendering in ChatMessage and ChatView streaming bubble Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,17 @@
|
||||
"""Add tool_calls JSONB column to messages."""
|
||||
|
||||
from alembic import op
|
||||
|
||||
revision = "0013"
|
||||
down_revision = "0012"
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
op.execute("""
|
||||
ALTER TABLE messages
|
||||
ADD COLUMN IF NOT EXISTS tool_calls JSONB
|
||||
""")
|
||||
|
||||
|
||||
def downgrade() -> None:
|
||||
op.drop_column("messages", "tool_calls")
|
||||
@@ -2,6 +2,7 @@
|
||||
import { computed } from "vue";
|
||||
import { renderMarkdown } from "@/utils/markdown";
|
||||
import { useSettingsStore } from "@/stores/settings";
|
||||
import ToolCallCard from "@/components/ToolCallCard.vue";
|
||||
import type { Message } from "@/types/chat";
|
||||
|
||||
const settingsStore = useSettingsStore();
|
||||
@@ -48,6 +49,13 @@ const roleLabel = computed(() => {
|
||||
</div>
|
||||
</div>
|
||||
<div class="message-content prose" v-html="rendered"></div>
|
||||
<div v-if="message.tool_calls?.length" class="tool-calls">
|
||||
<ToolCallCard
|
||||
v-for="(tc, i) in message.tool_calls"
|
||||
:key="i"
|
||||
:tool-call="tc"
|
||||
/>
|
||||
</div>
|
||||
<div
|
||||
v-if="message.context_note_id"
|
||||
class="context-badge"
|
||||
@@ -157,6 +165,12 @@ const roleLabel = computed(() => {
|
||||
color: #fff;
|
||||
border-color: var(--color-primary);
|
||||
}
|
||||
.tool-calls {
|
||||
display: flex;
|
||||
flex-wrap: wrap;
|
||||
gap: 0.3rem;
|
||||
margin-top: 0.4rem;
|
||||
}
|
||||
.context-badge {
|
||||
margin-top: 0.4rem;
|
||||
font-size: 0.75rem;
|
||||
|
||||
@@ -0,0 +1,124 @@
|
||||
<script setup lang="ts">
|
||||
import { computed } from "vue";
|
||||
import type { ToolCallRecord } from "@/types/chat";
|
||||
|
||||
const props = defineProps<{
|
||||
toolCall: ToolCallRecord;
|
||||
}>();
|
||||
|
||||
const label = computed(() => {
|
||||
if (!props.toolCall.result.success) return "Error";
|
||||
switch (props.toolCall.result.type) {
|
||||
case "task":
|
||||
return "Created task";
|
||||
case "note":
|
||||
return "Created note";
|
||||
case "search":
|
||||
return "Searched notes";
|
||||
default:
|
||||
return "Tool call";
|
||||
}
|
||||
});
|
||||
|
||||
const title = computed(() => {
|
||||
const data = props.toolCall.result.data;
|
||||
if (!data) return null;
|
||||
if (typeof data.title === "string") return data.title;
|
||||
return null;
|
||||
});
|
||||
|
||||
const linkTo = computed(() => {
|
||||
const data = props.toolCall.result.data;
|
||||
if (!data || typeof data.id !== "number") return null;
|
||||
return `/notes/${data.id}`;
|
||||
});
|
||||
|
||||
const searchResults = computed(() => {
|
||||
const data = props.toolCall.result.data;
|
||||
if (!data || props.toolCall.result.type !== "search") return null;
|
||||
const results = data.results as Array<{ id: number; title: string; type: string }> | undefined;
|
||||
return results && results.length > 0 ? results : null;
|
||||
});
|
||||
</script>
|
||||
|
||||
<template>
|
||||
<div class="tool-call-card" :class="{ error: toolCall.status === 'error' }">
|
||||
<span class="tool-label">{{ label }}</span>
|
||||
<template v-if="toolCall.status === 'error'">
|
||||
<span class="tool-error">{{ toolCall.result.error }}</span>
|
||||
</template>
|
||||
<template v-else-if="searchResults">
|
||||
<span class="tool-search-info">{{ (toolCall.result.data?.total as number) ?? 0 }} found</span>
|
||||
<div class="tool-search-results">
|
||||
<router-link
|
||||
v-for="r in searchResults"
|
||||
:key="r.id"
|
||||
:to="`/notes/${r.id}`"
|
||||
class="tool-search-item"
|
||||
>
|
||||
{{ r.title || "Untitled" }}
|
||||
</router-link>
|
||||
</div>
|
||||
</template>
|
||||
<template v-else-if="linkTo">
|
||||
<router-link :to="linkTo" class="tool-link">{{ title }}</router-link>
|
||||
</template>
|
||||
</div>
|
||||
</template>
|
||||
|
||||
<style scoped>
|
||||
.tool-call-card {
|
||||
display: inline-flex;
|
||||
flex-wrap: wrap;
|
||||
align-items: center;
|
||||
gap: 0.35rem;
|
||||
background: var(--color-bg-secondary);
|
||||
border: 1px solid var(--color-border);
|
||||
border-radius: 12px;
|
||||
padding: 0.3rem 0.6rem;
|
||||
font-size: 0.8rem;
|
||||
margin-top: 0.4rem;
|
||||
}
|
||||
.tool-call-card.error {
|
||||
border-color: var(--color-danger, #e74c3c);
|
||||
}
|
||||
.tool-label {
|
||||
font-weight: 600;
|
||||
color: var(--color-text-muted);
|
||||
text-transform: uppercase;
|
||||
letter-spacing: 0.03em;
|
||||
font-size: 0.7rem;
|
||||
}
|
||||
.tool-link {
|
||||
color: var(--color-primary);
|
||||
text-decoration: none;
|
||||
}
|
||||
.tool-link:hover {
|
||||
text-decoration: underline;
|
||||
}
|
||||
.tool-error {
|
||||
color: var(--color-danger, #e74c3c);
|
||||
}
|
||||
.tool-search-info {
|
||||
color: var(--color-text-muted);
|
||||
}
|
||||
.tool-search-results {
|
||||
display: flex;
|
||||
flex-wrap: wrap;
|
||||
gap: 0.25rem;
|
||||
width: 100%;
|
||||
}
|
||||
.tool-search-item {
|
||||
color: var(--color-primary);
|
||||
text-decoration: none;
|
||||
font-size: 0.8rem;
|
||||
}
|
||||
.tool-search-item:hover {
|
||||
text-decoration: underline;
|
||||
}
|
||||
.tool-search-item:not(:last-child)::after {
|
||||
content: ",";
|
||||
color: var(--color-text-muted);
|
||||
margin-right: 0.1rem;
|
||||
}
|
||||
</style>
|
||||
@@ -17,6 +17,7 @@ import type {
|
||||
OllamaStatus,
|
||||
RunningModel,
|
||||
SendMessageResponse,
|
||||
ToolCallRecord,
|
||||
} from "@/types/chat";
|
||||
|
||||
export const useChatStore = defineStore("chat", () => {
|
||||
@@ -26,6 +27,7 @@ export const useChatStore = defineStore("chat", () => {
|
||||
const loading = ref(false);
|
||||
const streaming = ref(false);
|
||||
const streamingContent = ref("");
|
||||
const streamingToolCalls = ref<ToolCallRecord[]>([]);
|
||||
const lastContextMeta = ref<ContextMeta | null>(null);
|
||||
const models = ref<OllamaModel[]>([]);
|
||||
const runningModels = ref<RunningModel[]>([]);
|
||||
@@ -187,6 +189,12 @@ export const useChatStore = defineStore("chat", () => {
|
||||
case "chunk":
|
||||
streamingContent.value += event.data.chunk as string;
|
||||
break;
|
||||
case "tool_call":
|
||||
streamingToolCalls.value = [
|
||||
...streamingToolCalls.value,
|
||||
event.data.tool_call as ToolCallRecord,
|
||||
];
|
||||
break;
|
||||
case "done":
|
||||
gotDone = true;
|
||||
{
|
||||
@@ -196,12 +204,16 @@ export const useChatStore = defineStore("chat", () => {
|
||||
role: "assistant",
|
||||
content: streamingContent.value,
|
||||
context_note_id: null,
|
||||
tool_calls: streamingToolCalls.value.length
|
||||
? [...streamingToolCalls.value]
|
||||
: null,
|
||||
created_at: new Date().toISOString(),
|
||||
};
|
||||
if (currentConversation.value?.id === convId) {
|
||||
currentConversation.value.messages.push(assistantMsg);
|
||||
}
|
||||
streamingContent.value = "";
|
||||
streamingToolCalls.value = [];
|
||||
streaming.value = false;
|
||||
|
||||
// Update conversation in list
|
||||
@@ -216,6 +228,7 @@ export const useChatStore = defineStore("chat", () => {
|
||||
gotDone = true;
|
||||
streaming.value = false;
|
||||
streamingContent.value = "";
|
||||
streamingToolCalls.value = [];
|
||||
useToastStore().show(
|
||||
"Chat error: " + (event.data.error as string),
|
||||
"error",
|
||||
@@ -252,6 +265,7 @@ export const useChatStore = defineStore("chat", () => {
|
||||
|
||||
streaming.value = false;
|
||||
streamingContent.value = "";
|
||||
streamingToolCalls.value = [];
|
||||
}
|
||||
|
||||
async function cancelGeneration() {
|
||||
@@ -357,6 +371,7 @@ export const useChatStore = defineStore("chat", () => {
|
||||
loading,
|
||||
streaming,
|
||||
streamingContent,
|
||||
streamingToolCalls,
|
||||
lastContextMeta,
|
||||
models,
|
||||
runningModels,
|
||||
|
||||
@@ -1,3 +1,10 @@
|
||||
export interface ToolCallRecord {
|
||||
function: string;
|
||||
arguments: Record<string, unknown>;
|
||||
result: { success: boolean; type?: string; data?: Record<string, unknown>; error?: string };
|
||||
status: "success" | "error";
|
||||
}
|
||||
|
||||
export interface Message {
|
||||
id: number;
|
||||
conversation_id: number;
|
||||
@@ -5,6 +12,7 @@ export interface Message {
|
||||
content: string;
|
||||
status?: "complete" | "generating" | "error";
|
||||
context_note_id: number | null;
|
||||
tool_calls?: ToolCallRecord[] | null;
|
||||
created_at: string;
|
||||
}
|
||||
|
||||
|
||||
@@ -7,6 +7,7 @@ import { apiGet } from "@/api/client";
|
||||
import { renderMarkdown } from "@/utils/markdown";
|
||||
import ChatMessage from "@/components/ChatMessage.vue";
|
||||
import ModelSelector from "@/components/ModelSelector.vue";
|
||||
import ToolCallCard from "@/components/ToolCallCard.vue";
|
||||
import type { Note } from "@/types/note";
|
||||
|
||||
const route = useRoute();
|
||||
@@ -400,6 +401,13 @@ function excludeAutoNote(noteId: number) {
|
||||
<div class="message-header">
|
||||
<span class="role-label">{{ settingsStore.assistantName }}</span>
|
||||
</div>
|
||||
<div v-if="store.streamingToolCalls.length" class="streaming-tool-calls">
|
||||
<ToolCallCard
|
||||
v-for="(tc, i) in store.streamingToolCalls"
|
||||
:key="i"
|
||||
:tool-call="tc"
|
||||
/>
|
||||
</div>
|
||||
<div class="message-content prose" v-html="streamingRendered"></div>
|
||||
<span class="typing-indicator"></span>
|
||||
</div>
|
||||
@@ -690,6 +698,12 @@ function excludeAutoNote(noteId: number) {
|
||||
word-break: break-word;
|
||||
}
|
||||
|
||||
.streaming-tool-calls {
|
||||
display: flex;
|
||||
flex-wrap: wrap;
|
||||
gap: 0.3rem;
|
||||
margin-bottom: 0.4rem;
|
||||
}
|
||||
.typing-indicator {
|
||||
display: inline-block;
|
||||
width: 6px;
|
||||
|
||||
@@ -2,6 +2,7 @@ from datetime import datetime, timezone
|
||||
|
||||
from sqlalchemy import DateTime, ForeignKey, Index, Integer, Text
|
||||
from sqlalchemy import inspect
|
||||
from sqlalchemy.dialects.postgresql import JSONB
|
||||
from sqlalchemy.orm import Mapped, mapped_column, relationship
|
||||
|
||||
from fabledassistant.models import Base
|
||||
@@ -65,6 +66,7 @@ class Message(Base):
|
||||
context_note_id: Mapped[int | None] = mapped_column(
|
||||
Integer, ForeignKey("notes.id", ondelete="SET NULL"), nullable=True
|
||||
)
|
||||
tool_calls: Mapped[list | None] = mapped_column(JSONB, nullable=True)
|
||||
created_at: Mapped[datetime] = mapped_column(
|
||||
DateTime(timezone=True), default=lambda: datetime.now(timezone.utc)
|
||||
)
|
||||
@@ -83,5 +85,6 @@ class Message(Base):
|
||||
"content": self.content,
|
||||
"status": self.status,
|
||||
"context_note_id": self.context_note_id,
|
||||
"tool_calls": self.tool_calls,
|
||||
"created_at": self.created_at.isoformat(),
|
||||
}
|
||||
|
||||
@@ -4,6 +4,7 @@ Streams from Ollama into a GenerationBuffer, periodically flushing to DB.
|
||||
Runs independently of any HTTP connection.
|
||||
"""
|
||||
|
||||
import json
|
||||
import logging
|
||||
import time
|
||||
|
||||
@@ -12,8 +13,9 @@ from sqlalchemy import update
|
||||
from fabledassistant.models import async_session
|
||||
from fabledassistant.models.conversation import Message
|
||||
from fabledassistant.services.generation_buffer import GenerationBuffer, GenerationState
|
||||
from fabledassistant.services.llm import generate_completion, stream_chat
|
||||
from fabledassistant.services.llm import ChatChunk, generate_completion, stream_chat, stream_chat_with_tools
|
||||
from fabledassistant.services.chat import update_conversation_title
|
||||
from fabledassistant.services.tools import TOOL_DEFINITIONS, execute_tool
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
@@ -47,12 +49,20 @@ async def _generate_title(messages: list[dict], model: str) -> str:
|
||||
return title[:100] if title else ""
|
||||
|
||||
|
||||
async def _update_message(message_id: int, content: str, status: str) -> None:
|
||||
async def _update_message(
|
||||
message_id: int,
|
||||
content: str,
|
||||
status: str,
|
||||
tool_calls: list[dict] | None = None,
|
||||
) -> None:
|
||||
values: dict = {"content": content, "status": status}
|
||||
if tool_calls is not None:
|
||||
values["tool_calls"] = tool_calls
|
||||
async with async_session() as session:
|
||||
await session.execute(
|
||||
update(Message)
|
||||
.where(Message.id == message_id)
|
||||
.values(content=content, status=status)
|
||||
.values(**values)
|
||||
)
|
||||
await session.commit()
|
||||
|
||||
@@ -68,33 +78,94 @@ async def run_generation(
|
||||
user_content: str,
|
||||
) -> None:
|
||||
"""Stream LLM response into buffer with periodic DB flushes."""
|
||||
MAX_TOOL_ROUNDS = 5
|
||||
msg_id = buf.assistant_message_id
|
||||
|
||||
# Emit context event
|
||||
buf.append_event("context", {"context": context_meta})
|
||||
|
||||
last_flush = time.monotonic()
|
||||
all_tool_calls: list[dict] = []
|
||||
|
||||
try:
|
||||
cancelled = False
|
||||
async for chunk in stream_chat(messages, model):
|
||||
if buf.cancel_event.is_set():
|
||||
cancelled = True
|
||||
|
||||
for _round in range(MAX_TOOL_ROUNDS + 1):
|
||||
round_tool_calls: list[dict] = []
|
||||
|
||||
async for chunk in stream_chat_with_tools(messages, model, tools=TOOL_DEFINITIONS):
|
||||
if buf.cancel_event.is_set():
|
||||
cancelled = True
|
||||
break
|
||||
|
||||
if chunk.type == "content":
|
||||
buf.content_so_far += chunk.content
|
||||
buf.append_event("chunk", {"chunk": chunk.content})
|
||||
|
||||
# Periodic DB flush
|
||||
now = time.monotonic()
|
||||
if now - last_flush >= DB_FLUSH_INTERVAL:
|
||||
try:
|
||||
await _update_message(msg_id, buf.content_so_far, "generating")
|
||||
except Exception:
|
||||
logger.warning("Failed periodic flush for message %d", msg_id, exc_info=True)
|
||||
last_flush = now
|
||||
|
||||
elif chunk.type == "tool_calls" and chunk.tool_calls:
|
||||
# Process each tool call
|
||||
for tc in chunk.tool_calls:
|
||||
fn = tc.get("function", {})
|
||||
tool_name = fn.get("name", "")
|
||||
arguments = fn.get("arguments", {})
|
||||
|
||||
result = await execute_tool(user_id, tool_name, arguments)
|
||||
|
||||
tool_record = {
|
||||
"function": tool_name,
|
||||
"arguments": arguments,
|
||||
"result": result,
|
||||
"status": "success" if result.get("success") else "error",
|
||||
}
|
||||
round_tool_calls.append(tool_record)
|
||||
all_tool_calls.append(tool_record)
|
||||
|
||||
# Emit tool_call SSE event
|
||||
buf.append_event("tool_call", {"tool_call": tool_record})
|
||||
|
||||
if cancelled:
|
||||
break
|
||||
buf.content_so_far += chunk
|
||||
buf.append_event("chunk", {"chunk": chunk})
|
||||
|
||||
# Periodic DB flush
|
||||
now = time.monotonic()
|
||||
if now - last_flush >= DB_FLUSH_INTERVAL:
|
||||
try:
|
||||
await _update_message(msg_id, buf.content_so_far, "generating")
|
||||
except Exception:
|
||||
logger.warning("Failed periodic flush for message %d", msg_id, exc_info=True)
|
||||
last_flush = now
|
||||
# If no tool calls this round, the LLM gave its final text response
|
||||
if not round_tool_calls:
|
||||
break
|
||||
|
||||
# Final save (partial content on cancel is still valid)
|
||||
await _update_message(msg_id, buf.content_so_far, "complete")
|
||||
# Append assistant tool_call message and tool results to conversation
|
||||
# for the next round
|
||||
messages.append({
|
||||
"role": "assistant",
|
||||
"content": buf.content_so_far,
|
||||
"tool_calls": [
|
||||
{"function": {"name": tc["function"], "arguments": tc["arguments"]}}
|
||||
for tc in round_tool_calls
|
||||
],
|
||||
})
|
||||
|
||||
for tc in round_tool_calls:
|
||||
messages.append({
|
||||
"role": "tool",
|
||||
"content": json.dumps(tc["result"]),
|
||||
})
|
||||
|
||||
# Reset content for the next round (LLM will produce a new response)
|
||||
buf.content_so_far = ""
|
||||
|
||||
# Final save
|
||||
await _update_message(
|
||||
msg_id,
|
||||
buf.content_so_far,
|
||||
"complete",
|
||||
tool_calls=all_tool_calls if all_tool_calls else None,
|
||||
)
|
||||
|
||||
# Count non-system messages to decide on title generation
|
||||
non_system = [m for m in messages if m["role"] != "system"]
|
||||
|
||||
@@ -2,6 +2,8 @@ import json
|
||||
import logging
|
||||
import re
|
||||
from collections.abc import AsyncGenerator
|
||||
from dataclasses import dataclass, field
|
||||
from typing import Literal
|
||||
|
||||
import httpx
|
||||
|
||||
@@ -98,6 +100,55 @@ async def stream_chat(
|
||||
break
|
||||
|
||||
|
||||
@dataclass
|
||||
class ChatChunk:
|
||||
"""A chunk yielded by stream_chat_with_tools."""
|
||||
type: Literal["content", "tool_calls", "done"]
|
||||
content: str = ""
|
||||
tool_calls: list[dict] | None = None
|
||||
|
||||
|
||||
async def stream_chat_with_tools(
|
||||
messages: list[dict],
|
||||
model: str,
|
||||
tools: list[dict] | None = None,
|
||||
) -> AsyncGenerator[ChatChunk, None]:
|
||||
"""Stream chat completion from Ollama with tool support.
|
||||
|
||||
Yields ChatChunk objects. If the model returns tool_calls, a
|
||||
ChatChunk(type="tool_calls") is yielded. Always ends with
|
||||
ChatChunk(type="done").
|
||||
"""
|
||||
payload: dict = {"model": model, "messages": messages, "stream": True}
|
||||
if tools:
|
||||
payload["tools"] = tools
|
||||
async with httpx.AsyncClient(timeout=httpx.Timeout(1800.0, connect=30.0, read=300.0)) as client:
|
||||
async with client.stream(
|
||||
"POST",
|
||||
f"{Config.OLLAMA_URL}/api/chat",
|
||||
json=payload,
|
||||
) as resp:
|
||||
resp.raise_for_status()
|
||||
async for line in resp.aiter_lines():
|
||||
if not line.strip():
|
||||
continue
|
||||
data = json.loads(line)
|
||||
msg = data.get("message", {})
|
||||
|
||||
# Content chunks
|
||||
chunk = msg.get("content", "")
|
||||
if chunk:
|
||||
yield ChatChunk(type="content", content=chunk)
|
||||
|
||||
# Check for tool calls on done
|
||||
if data.get("done"):
|
||||
tool_calls = msg.get("tool_calls")
|
||||
if tool_calls:
|
||||
yield ChatChunk(type="tool_calls", tool_calls=tool_calls)
|
||||
yield ChatChunk(type="done")
|
||||
break
|
||||
|
||||
|
||||
async def generate_completion(messages: list[dict], model: str) -> str:
|
||||
"""Non-streaming chat completion, returns full response text."""
|
||||
async with httpx.AsyncClient(timeout=httpx.Timeout(1800.0, connect=30.0, read=300.0)) as client:
|
||||
@@ -166,11 +217,17 @@ async def build_context(
|
||||
which notes were included as context.
|
||||
"""
|
||||
exclude_set = set(exclude_note_ids or [])
|
||||
from datetime import date as date_type
|
||||
assistant_name = await get_setting(user_id, "assistant_name", "Fable")
|
||||
today = date_type.today().isoformat()
|
||||
system_parts = [
|
||||
f"You are a helpful assistant named {assistant_name}, integrated into a note-taking and task-tracking app called Fabled Assistant. "
|
||||
"Help users with their notes, tasks, and general questions. "
|
||||
"When note context is provided, use it to give relevant answers."
|
||||
"When note context is provided, use it to give relevant answers. "
|
||||
f"Today's date is {today}. "
|
||||
"You have tools available to create tasks, create notes, and search the user's notes. "
|
||||
"Use them when the user asks you to create, add, or find tasks and notes. "
|
||||
"For relative dates like 'Friday' or 'next week', resolve them to YYYY-MM-DD format."
|
||||
]
|
||||
|
||||
context_meta: dict = {
|
||||
|
||||
@@ -0,0 +1,159 @@
|
||||
"""Tool definitions and executor for LLM tool calling."""
|
||||
|
||||
import logging
|
||||
from datetime import date, datetime
|
||||
|
||||
from fabledassistant.services.notes import create_note, list_notes
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
TOOL_DEFINITIONS = [
|
||||
{
|
||||
"type": "function",
|
||||
"function": {
|
||||
"name": "create_task",
|
||||
"description": "Create a new task for the user. Use this when the user asks you to add a task, todo, or reminder.",
|
||||
"parameters": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"title": {
|
||||
"type": "string",
|
||||
"description": "The task title",
|
||||
},
|
||||
"body": {
|
||||
"type": "string",
|
||||
"description": "Optional task description or details",
|
||||
},
|
||||
"due_date": {
|
||||
"type": "string",
|
||||
"description": "Optional due date in YYYY-MM-DD format",
|
||||
},
|
||||
"priority": {
|
||||
"type": "string",
|
||||
"enum": ["none", "low", "medium", "high"],
|
||||
"description": "Task priority level",
|
||||
},
|
||||
},
|
||||
"required": ["title"],
|
||||
},
|
||||
},
|
||||
},
|
||||
{
|
||||
"type": "function",
|
||||
"function": {
|
||||
"name": "create_note",
|
||||
"description": "Create a new note for the user. Use this when the user asks you to write down, save, or record something as a note.",
|
||||
"parameters": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"title": {
|
||||
"type": "string",
|
||||
"description": "The note title",
|
||||
},
|
||||
"body": {
|
||||
"type": "string",
|
||||
"description": "The note content in markdown",
|
||||
},
|
||||
},
|
||||
"required": ["title"],
|
||||
},
|
||||
},
|
||||
},
|
||||
{
|
||||
"type": "function",
|
||||
"function": {
|
||||
"name": "search_notes",
|
||||
"description": "Search the user's notes and tasks. Use this when the user asks about their existing notes, tasks, or wants to find something they've written.",
|
||||
"parameters": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"query": {
|
||||
"type": "string",
|
||||
"description": "Search query to find notes/tasks",
|
||||
},
|
||||
},
|
||||
"required": ["query"],
|
||||
},
|
||||
},
|
||||
},
|
||||
]
|
||||
|
||||
|
||||
def _parse_due_date(value: str | None) -> date | None:
|
||||
"""Parse a due date string, returning None on failure."""
|
||||
if not value:
|
||||
return None
|
||||
try:
|
||||
return datetime.strptime(value, "%Y-%m-%d").date()
|
||||
except (ValueError, TypeError):
|
||||
logger.warning("Invalid due_date format: %s", value)
|
||||
return None
|
||||
|
||||
|
||||
async def execute_tool(user_id: int, tool_name: str, arguments: dict) -> dict:
|
||||
"""Execute a tool call and return the result."""
|
||||
try:
|
||||
if tool_name == "create_task":
|
||||
note = await create_note(
|
||||
user_id=user_id,
|
||||
title=arguments.get("title", "Untitled Task"),
|
||||
body=arguments.get("body", ""),
|
||||
status="todo",
|
||||
priority=arguments.get("priority", "none"),
|
||||
due_date=_parse_due_date(arguments.get("due_date")),
|
||||
)
|
||||
return {
|
||||
"success": True,
|
||||
"type": "task",
|
||||
"data": {
|
||||
"id": note.id,
|
||||
"title": note.title,
|
||||
"status": note.status,
|
||||
"priority": note.priority,
|
||||
"due_date": str(note.due_date) if note.due_date else None,
|
||||
},
|
||||
}
|
||||
|
||||
elif tool_name == "create_note":
|
||||
note = await create_note(
|
||||
user_id=user_id,
|
||||
title=arguments.get("title", "Untitled Note"),
|
||||
body=arguments.get("body", ""),
|
||||
)
|
||||
return {
|
||||
"success": True,
|
||||
"type": "note",
|
||||
"data": {
|
||||
"id": note.id,
|
||||
"title": note.title,
|
||||
},
|
||||
}
|
||||
|
||||
elif tool_name == "search_notes":
|
||||
query = arguments.get("query", "")
|
||||
notes, total = await list_notes(user_id=user_id, q=query, limit=5)
|
||||
results = []
|
||||
for n in notes:
|
||||
results.append({
|
||||
"id": n.id,
|
||||
"title": n.title,
|
||||
"type": "task" if n.status is not None else "note",
|
||||
"status": n.status,
|
||||
"preview": (n.body[:200] + "...") if n.body and len(n.body) > 200 else (n.body or ""),
|
||||
})
|
||||
return {
|
||||
"success": True,
|
||||
"type": "search",
|
||||
"data": {
|
||||
"query": query,
|
||||
"total": total,
|
||||
"results": results,
|
||||
},
|
||||
}
|
||||
|
||||
else:
|
||||
return {"success": False, "error": f"Unknown tool: {tool_name}"}
|
||||
|
||||
except Exception as e:
|
||||
logger.exception("Tool execution failed: %s", tool_name)
|
||||
return {"success": False, "error": str(e)}
|
||||
+21
-10
@@ -12,7 +12,7 @@
|
||||
> Include file-level details in the commit body when the change is non-trivial.
|
||||
|
||||
## Last Updated
|
||||
2026-02-14 — Phase 6.1: Model status colors, timeout tuning, SSE error feedback
|
||||
2026-02-14 — Phase 7: LLM tool calling (create tasks, create notes, search notes from chat)
|
||||
|
||||
## Project Overview
|
||||
Fabled Assistant is a self-hosted note-taking and task-tracking application with
|
||||
@@ -202,10 +202,11 @@ for AI-assisted features.
|
||||
### Messages
|
||||
- `id` (int PK), `conversation_id` (FK to conversations, CASCADE), `role` (str: system/user/assistant),
|
||||
`content` (str), `status` (str, default `'complete'` — `complete`/`generating`/`error`),
|
||||
`context_note_id` (nullable FK to notes, SET NULL), `created_at`
|
||||
`context_note_id` (nullable FK to notes, SET NULL), `tool_calls` (nullable JSONB), `created_at`
|
||||
- Index on `conversation_id`
|
||||
- `context_note_id` tracks which note was attached as context when message was sent
|
||||
- `to_dict()` returns: `id`, `conversation_id`, `role`, `content`, `status`, `context_note_id`, `created_at`
|
||||
- `tool_calls` stores an array of tool call records `[{function, arguments, result, status}]` for assistant messages that used tools
|
||||
- `to_dict()` returns: `id`, `conversation_id`, `role`, `content`, `status`, `context_note_id`, `tool_calls`, `created_at`
|
||||
|
||||
## Project Structure (Current)
|
||||
```
|
||||
@@ -230,7 +231,8 @@ fabledassistant/
|
||||
│ ├── 0009_add_message_status.py # Add status column to messages table (default 'complete')
|
||||
│ ├── 0010_add_app_logs_table.py # App logs table for audit, usage, and error logging
|
||||
│ ├── 0011_add_password_reset_tokens.py # Password reset tokens table
|
||||
│ └── 0012_add_invitation_tokens.py # Invitation tokens table
|
||||
│ ├── 0012_add_invitation_tokens.py # Invitation tokens table
|
||||
│ └── 0013_add_tool_calls_to_messages.py # Add tool_calls JSONB column to messages
|
||||
├── src/
|
||||
│ └── fabledassistant/
|
||||
│ ├── __init__.py
|
||||
@@ -258,10 +260,11 @@ fabledassistant/
|
||||
│ │ ├── auth.py # Auth business logic: hash_password, verify_password, create_user, authenticate, get_user_by_id, is_registration_open, list_users, delete_user, set_registration_open, password reset tokens, invitation tokens (create, validate, register_with_invitation, list_pending, revoke)
|
||||
│ │ ├── backup.py # Backup/restore: export_full_backup, export_user_backup, restore_full_backup
|
||||
│ │ ├── notes.py # CRUD with user_id isolation, is_task filter, convert, backlinks, search_notes_for_context
|
||||
│ │ ├── llm.py # Ollama interaction: build_context with user_id, streaming, URL fetching
|
||||
│ │ ├── llm.py # Ollama interaction: build_context with user_id, streaming (stream_chat + stream_chat_with_tools), ChatChunk dataclass, URL fetching
|
||||
│ │ ├── chat.py # Conversation CRUD with user_id isolation, add_message, save/summarize as note (LLM-titled, chat-tagged)
|
||||
│ │ ├── generation_buffer.py # In-memory SSE event buffer with cancel_event, reconnect support, auto-cleanup; supports chat (int keys) and assist (string keys)
|
||||
│ │ ├── generation_task.py # Background asyncio tasks: run_generation (chat, DB flush, titles) + run_assist_generation (lightweight, no DB)
|
||||
│ │ ├── generation_task.py # Background asyncio tasks: run_generation (chat, DB flush, titles, tool loop) + run_assist_generation (lightweight, no DB)
|
||||
│ │ ├── tools.py # LLM tool definitions (create_task, create_note, search_notes) + execute_tool dispatcher
|
||||
│ │ ├── settings.py # Settings CRUD with user_id isolation: get_setting, set_setting, set_settings_batch, get_all_settings
|
||||
│ │ ├── logging.py # App logging: log_audit, log_usage, log_error, get_logs, get_log_stats, delete_old_logs, start_log_retention_loop
|
||||
│ │ ├── email.py # SMTP email service: get_smtp_config, is_smtp_configured, send_email, send_test_email
|
||||
@@ -295,7 +298,7 @@ fabledassistant/
|
||||
│ ├── types/
|
||||
│ │ ├── auth.ts # User interface, AuthStatus interface
|
||||
│ │ ├── note.ts # Note interface (with status, priority, due_date, is_task) + TaskStatus, TaskPriority types + NoteListResponse
|
||||
│ │ ├── chat.ts # Message, Conversation, ConversationDetail, ContextMeta, OllamaModel, RunningModel, OllamaStatus interfaces
|
||||
│ │ ├── chat.ts # ToolCallRecord, Message, Conversation, ConversationDetail, ContextMeta, OllamaModel, RunningModel, OllamaStatus interfaces
|
||||
│ │ ├── settings.ts # AppSettings interface, ModelInfo interface (name, description, size, bestFor, category)
|
||||
│ │ └── task.ts # Task = re-export of Note; TaskListResponse
|
||||
│ ├── extensions/
|
||||
@@ -328,7 +331,8 @@ fabledassistant/
|
||||
│ │ ├── ChatPanel.vue # Slide-out chat panel (right side overlay, receives contextNoteId prop), bubble-style messages, floating dark input, note picker, context pills with promote/exclude
|
||||
│ │ ├── ModelSelector.vue # Model dropdown (v-model pattern): fetches installed + running models, hot/cold indicators
|
||||
│ │ ├── DashboardChatInput.vue # Inline chat bar: ModelSelector + note picker + textarea + send button; emits submit event
|
||||
│ │ ├── ChatMessage.vue # Message bubble: markdown rendering, configurable assistant name label, "Save as Note" action on assistant messages
|
||||
│ │ ├── ToolCallCard.vue # Compact card for tool call results (created task/note link, search results, errors)
|
||||
│ │ ├── ChatMessage.vue # Message bubble: markdown rendering, tool call cards, configurable assistant name label, "Save as Note" action on assistant messages
|
||||
│ │ ├── NoteCard.vue # Card with rendered markdown preview (v-html), TagPill, tag-click emit, hover edit button
|
||||
│ │ ├── TaskCard.vue # Card with rendered preview (body not description), StatusBadge (clickable), PriorityBadge, due date, tags, hover edit button
|
||||
│ │ ├── StatusBadge.vue # Color-coded status badge, optional clickable cycling
|
||||
@@ -422,7 +426,7 @@ container startup.
|
||||
|
||||
### Migration Chain
|
||||
```
|
||||
0001_create_notes_table.py → 0002_create_tasks_table.py → 0003_task_note_companion.py → 0004_merge_tasks_into_notes.py → 0005_add_chat_tables.py → 0006_add_settings_table.py → 0007_add_title_and_updated_at_indexes.py → 0008_add_users_and_user_id.py → 0009_add_message_status.py → 0010_add_app_logs_table.py → 0011_add_password_reset_tokens.py → 0012_add_invitation_tokens.py
|
||||
0001_create_notes_table.py → 0002_create_tasks_table.py → 0003_task_note_companion.py → 0004_merge_tasks_into_notes.py → 0005_add_chat_tables.py → 0006_add_settings_table.py → 0007_add_title_and_updated_at_indexes.py → 0008_add_users_and_user_id.py → 0009_add_message_status.py → 0010_add_app_logs_table.py → 0011_add_password_reset_tokens.py → 0012_add_invitation_tokens.py → 0013_add_tool_calls_to_messages.py
|
||||
```
|
||||
|
||||
### How Migrations Run
|
||||
@@ -531,6 +535,13 @@ When adding a new migration, follow these conventions:
|
||||
- Ollama configured with `OLLAMA_MAX_LOADED_MODELS=2` and `OLLAMA_KEEP_ALIVE=30m`
|
||||
- Timeout tuning: connect timeout 30s (cold model loading), warm timeout 300s, pull timeout 1800s
|
||||
- SSE reconnection failure shows error toast instead of silently recovering
|
||||
- **LLM tool calling:** Models with tool support can create tasks, create notes, and search notes
|
||||
on behalf of the user during chat. Multi-round tool loop (max 5 rounds) allows the LLM to
|
||||
execute tools and then produce a natural language response incorporating results. Tool call
|
||||
results are persisted in message `tool_calls` JSONB column and rendered as compact ToolCallCard
|
||||
components (linked titles for created items, search result lists). SSE emits `tool_call` events
|
||||
for real-time rendering during streaming. System prompt includes today's date for relative date
|
||||
resolution. Graceful degradation: models without tool support respond normally.
|
||||
|
||||
### Authentication & User Management
|
||||
- Session cookie auth with bcrypt, first-user-is-admin, orphaned data claiming
|
||||
@@ -571,7 +582,7 @@ When adding a new migration, follow these conventions:
|
||||
- Multi-stage Dockerfile: Node build → Python runtime, auto-migration on startup
|
||||
- Docker Compose (dev) + Docker Swarm production stack with secrets, overlay networks, health checks
|
||||
- Quart serves Vue SPA from static + REST API under `/api/`; 404 handler for SPA routing
|
||||
- Alembic migrations: 12 migrations, all raw SQL with idempotency guards (IF NOT EXISTS, DO $$ BEGIN...EXCEPTION)
|
||||
- Alembic migrations: 13 migrations, all raw SQL with idempotency guards (IF NOT EXISTS, DO $$ BEGIN...EXCEPTION)
|
||||
- Config from env vars + Docker secrets file support (`_read_secret`)
|
||||
|
||||
## Development Workflow
|
||||
|
||||
Reference in New Issue
Block a user