diff --git a/docs/superpowers/specs/2026-04-06-research-multi-note-design.md b/docs/superpowers/specs/2026-04-06-research-multi-note-design.md new file mode 100644 index 0000000..2ec0b16 --- /dev/null +++ b/docs/superpowers/specs/2026-04-06-research-multi-note-design.md @@ -0,0 +1,162 @@ +# Research Pipeline — Multi-Note Redesign + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Replace the single monolithic research note with a set of focused, topic-driven notes plus an index note that links them — making research output browsable, TTS-friendly, and well-organized. + +**Architecture:** Two new LLM calls (outline generation + N parallel section syntheses) replace the single large synthesis call. Public API unchanged — callers receive the index note. Fallback to single-note behavior on any outline failure. + +**Tech Stack:** Python/Quart backend, existing `research.py` service, asyncio.gather for parallelism. + +--- + +## Problem + +The current pipeline synthesizes one note with a minimum of 2500 words and 6 sections. This creates: +- Notes too large to read or listen to comfortably +- No way to navigate directly to a specific sub-topic +- TTS failures on long prose (8000-char route limit, unbounded sentence buffers) + +--- + +## Pipeline Flow + +Public signature unchanged: +```python +async def run_research_pipeline( + topic: str, + user_id: int, + model: str, + buf=None, + project_id: int | None = None, +) -> Note: # returns the index note +``` + +Execution order: + +``` +1. Generate sub-queries (unchanged) +2. Search + fetch sources (unchanged) +3. Generate topic outline (NEW — one LLM call → 3–7 section dicts) +4. Synthesize each section note (NEW — parallelized via asyncio.gather) +5. Create all section notes in DB (sequential, tagged ["research"], same project_id) +6. Create index note (NEW — links all sections) +7. Return index note +``` + +Status messages via `buf.append_event("status", ...)`: +- `"Generating outline…"` +- `"Writing: [Section Title]…"` (one per section, emitted before synthesis starts) +- `"Saving [N] notes…"` + +No note content is streamed into chat. After the tool call resolves, the LLM writes a brief conversational summary citing the index note title and section count. + +--- + +## Outline Generation + +New function: `_generate_outline(topic, sources, model) -> list[dict]` + +Sends all fetched sources to the model with a prompt requesting a JSON array: + +```json +[ + {"title": "Quantum Entanglement: Mechanisms", "focus": "How entanglement works at the physical level"}, + {"title": "Quantum Computing Hardware", "focus": "Ion traps, superconducting qubits, photonic approaches"} +] +``` + +**Prompt requirements:** +- Produce 3–7 sections covering distinct aspects of the topic +- Titles must work as standalone note titles (no "Overview" or "Introduction" generics) +- No overlap between sections +- `focus` is one sentence describing what this section should specifically cover + +**Guardrails:** +- Fewer than 3 sections parsed → fall back to single-note synthesis +- JSON parse failure → fall back to single-note synthesis +- More than 8 sections → truncate to 8 + +**Model params:** `max_tokens=400, num_ctx=16384` (outline is short) + +--- + +## Section Synthesis + +New function: `_synthesize_section(section_title, section_focus, sources, model) -> tuple[str, str]` + +Returns `(title, body_markdown)`. + +All sections receive all fetched sources. The `section_focus` field in the prompt directs the model to draw only what's relevant to that section's scope. + +**Prompt requirements:** +- 300–600 words of substantive prose +- Do NOT include a `# Title` heading (title is set separately) +- End with a brief `## Sources` list of relevant URLs from the provided sources +- Focus strictly on `section_focus` — ignore source material outside that scope + +**Model params:** `num_predict=2048, num_ctx=16384` (reduced from 8192 — sufficient for 600 words, prevents rambling) + +**Parallelism:** All section synthesis calls run via `asyncio.gather`. Wall-clock time stays close to a single synthesis call despite producing N notes. + +--- + +## Note Creation and Index Note + +**Section notes:** +- Tags: `["research"]` +- `project_id`: same as passed to pipeline (or None) +- Title: from outline `title` field +- Created sequentially (avoids DB contention) + +**Index note:** +- Tags: `["research", "research-index"]` +- `project_id`: same as section notes +- Title: `"Research: [topic]"` +- Created last (after all section notes exist) + +**Index note body format:** +```markdown +Research overview for **[topic]** — [YYYY-MM-DD] + +Generated from [N] web sources across [M] sections. + +## Sections + +- **[Section 1 Title]** — [focus sentence] +- **[Section 2 Title]** — [focus sentence] +... + +*Search for any section title to read it.* +``` + +The index note is what `run_research_pipeline` returns. The existing `research_topic` tool handler uses `note.id` and `note.title` — both remain valid with the index note. + +--- + +## Error Handling + +| Scenario | Behaviour | +|---|---| +| Outline generation raises | Fall back to single-note synthesis (current behaviour) | +| Outline JSON unparseable | Fall back to single-note synthesis | +| Outline returns < 3 sections | Fall back to single-note synthesis | +| Outline returns > 8 sections | Truncate to 8, continue | +| A section synthesis raises | Log warning, skip that section; continue with remaining | +| All section syntheses fail | Fall back to single-note synthesis | +| A section note DB save fails | Log warning, skip from index; index note still created | +| No sources fetched | Raise `ValueError` as today — unchanged | + +The fallback in every case is the current single-note pipeline. Research never silently produces nothing. + +--- + +## What Is NOT Changing + +- Public function signature of `run_research_pipeline` +- Sub-query generation (`_generate_sub_queries`) +- SearXNG search and URL fetching +- `_search_searxng`, `_search_searxng_images`, `fetch_url_content` +- The `research_topic` tool definition and handler in `tools.py` +- The `quick_capture` research path +- Any frontend component diff --git a/frontend/src/components/AppHeader.vue b/frontend/src/components/AppHeader.vue index 62adce0..013fcb4 100644 --- a/frontend/src/components/AppHeader.vue +++ b/frontend/src/components/AppHeader.vue @@ -77,6 +77,7 @@ router.afterEach(() => { Chat Briefing Calendar + News Tasks Projects diff --git a/frontend/src/components/ChatPanel.vue b/frontend/src/components/ChatPanel.vue index 3be41ef..48c05e2 100644 --- a/frontend/src/components/ChatPanel.vue +++ b/frontend/src/components/ChatPanel.vue @@ -299,12 +299,22 @@ defineExpose({ focus, prefill, send })
- + > + +
+ New Briefing Update +
+ + @@ -523,6 +533,28 @@ defineExpose({ focus, prefill, send }) justify-content: flex-end; } +/* Briefing slot separator */ +.briefing-slot-separator { + display: flex; + align-items: center; + gap: 0.75rem; + margin: 1.25rem 0 0.5rem; + color: var(--color-text-muted, #888); + font-size: 0.7rem; + font-weight: 600; + letter-spacing: 0.08em; + text-transform: uppercase; + opacity: 0.7; +} +.briefing-slot-separator::before, +.briefing-slot-separator::after { + content: ''; + flex: 1; + height: 1px; + background: var(--color-border, #333); + opacity: 0.5; +} + /* Context sidebar */ .context-sidebar { width: 200px; diff --git a/frontend/src/composables/useStreamingTts.ts b/frontend/src/composables/useStreamingTts.ts index dda7084..80c9ed2 100644 --- a/frontend/src/composables/useStreamingTts.ts +++ b/frontend/src/composables/useStreamingTts.ts @@ -86,11 +86,21 @@ export function useStreamingTts(options: UseStreamingTtsOptions): UseStreamingTt try { blob = await synthesiseSpeech(stripped) } catch (e) { - console.warn('[StreamingTTS] Synthesis failed, retrying sentence', { sentence: stripped, error: e }) + const errMsg = e instanceof Error ? e.message : String(e) + console.warn('[StreamingTTS] Synthesis failed, retrying', { + chars: stripped.length, + preview: stripped.slice(0, 80), + error: errMsg, + }) try { blob = await synthesiseSpeech(stripped) } catch (e2) { - console.warn('[StreamingTTS] Retry also failed, skipping sentence', { sentence: stripped, error: e2 }) + const errMsg2 = e2 instanceof Error ? e2.message : String(e2) + console.warn('[StreamingTTS] Retry failed, sentence dropped', { + chars: stripped.length, + preview: stripped.slice(0, 80), + error: errMsg2, + }) } } finally { pendingCount.value-- diff --git a/frontend/src/types/chat.ts b/frontend/src/types/chat.ts index 9d97812..162dd99 100644 --- a/frontend/src/types/chat.ts +++ b/frontend/src/types/chat.ts @@ -28,6 +28,7 @@ export interface Message { context_note_id: number | null; context_note_title?: string | null; tool_calls?: ToolCallRecord[] | null; + metadata?: Record | null; created_at: string; timing?: GenerationTiming; thinking?: string; diff --git a/src/fabledassistant/routes/voice.py b/src/fabledassistant/routes/voice.py index e4bb98b..b0ed890 100644 --- a/src/fabledassistant/routes/voice.py +++ b/src/fabledassistant/routes/voice.py @@ -117,7 +117,12 @@ async def synthesise_speech(): if not text: return jsonify({"error": "text is required"}), 400 - if len(text) > 8000: + char_count = len(text) + if char_count > 8000: + logger.warning( + "TTS request rejected: text too long (%d chars, limit 8000). Preview: %r", + char_count, text[:120], + ) return jsonify({"error": "text too long (max 8000 characters)"}), 400 voice = str(data.get("voice", "af_heart")) @@ -154,11 +159,29 @@ async def synthesise_speech(): except Exception: pass + blend_desc = f"blend({len(voice_blend)} voices)" if voice_blend else voice + logger.info("TTS synthesis start: %d chars, voice=%s, speed=%.2f", char_count, blend_desc, speed) + t0 = time.monotonic() try: wav_bytes = await synthesise(text, voice=voice, speed=speed, voice_blend=voice_blend) except Exception: - logger.exception("TTS synthesis failed") + logger.exception( + "TTS synthesis failed: %d chars, voice=%s. Preview: %r", + char_count, blend_desc, text[:120], + ) return jsonify({"error": "Synthesis failed"}), 500 + duration_ms = round((time.monotonic() - t0) * 1000) + if not wav_bytes: + logger.warning( + "TTS synthesis returned empty audio: %d chars, voice=%s, %dms. Preview: %r", + char_count, blend_desc, duration_ms, text[:120], + ) + else: + logger.info( + "TTS synthesis complete: %d chars → %d bytes in %dms (voice=%s)", + char_count, len(wav_bytes), duration_ms, blend_desc, + ) + from quart import Response return Response(wav_bytes, mimetype="audio/wav") diff --git a/src/fabledassistant/services/llm.py b/src/fabledassistant/services/llm.py index 67abd71..178e501 100644 --- a/src/fabledassistant/services/llm.py +++ b/src/fabledassistant/services/llm.py @@ -607,6 +607,25 @@ async def build_context( f"\n\n--- Earlier Conversation ---\n{history_summary}\n--- End Earlier Conversation ---" ) + # Detect briefing conversation — used for both system prompt instruction and article injection + _is_briefing_conv = False + if conv_id is not None: + from fabledassistant.models import async_session as _async_session + from fabledassistant.models.conversation import Conversation as _Conversation + async with _async_session() as _sess: + _conv = await _sess.get(_Conversation, conv_id) + if _conv and getattr(_conv, "conversation_type", None) == "briefing": + _is_briefing_conv = True + + if _is_briefing_conv: + system_content += ( + "\n\nYou are in a briefing conversation. " + "The conversation history contains today's briefing — news stories, weather, and tasks. " + "When the user asks about a topic, person, or event from the briefing, answer directly " + "from the conversation history and the article context that follows. " + "Do NOT search the web for information that is already present in the briefing." + ) + context_meta: dict = { "context_note_id": None, "context_note_title": None, @@ -769,16 +788,10 @@ async def build_context( ) # Briefing article context for follow-up Q&A - if conv_id is not None: - from fabledassistant.models import async_session as _async_session - from fabledassistant.models.conversation import Conversation - - async with _async_session() as _sess: - _conv = await _sess.get(Conversation, conv_id) - if _conv and getattr(_conv, "conversation_type", None) == "briefing": - article_context = await _build_briefing_article_context(conv_id) - if article_context: - user_context_parts.append(article_context.strip()) + if _is_briefing_conv: + article_context = await _build_briefing_article_context(conv_id) # type: ignore[arg-type] + if article_context: + user_context_parts.append(article_context.strip()) # Build final user message — context prefix (if any) followed by the actual message if user_context_parts: diff --git a/src/fabledassistant/services/research.py b/src/fabledassistant/services/research.py index 0f6233b..7293da8 100644 --- a/src/fabledassistant/services/research.py +++ b/src/fabledassistant/services/research.py @@ -21,6 +21,106 @@ MAX_SYNTHESIS_SOURCES = 12 # deduplicated sources passed to synthesis LLM CHARS_PER_SOURCE = 2000 # content chars per source sent to synthesis +def _build_sources_block(sources: list[dict]) -> str: + """Format fetched sources into a text block for LLM prompts.""" + parts = [] + for i, s in enumerate(sources, 1): + content = (s.get("content") or s.get("snippet") or "")[:CHARS_PER_SOURCE] + parts.append( + f"[Source {i}] {s['title']}\nURL: {s['url']}\nSearch query: {s['query']}\n\n{content}" + ) + return "\n\n" + ("─" * 60) + "\n\n".join(parts) + + +async def _generate_outline(topic: str, sources: list[dict], model: str) -> list[dict]: + """Generate a topic outline from fetched research sources. + + Returns a list of {"title": str, "focus": str} dicts (3–8 entries). + Returns [] on failure — callers must fall back to single-note synthesis. + """ + import json as _json + + sources_block = _build_sources_block(sources) if sources else "(no sources)" + messages = [ + { + "role": "system", + "content": ( + "You are a research organizer. Given research sources on a topic, produce a JSON array " + "of section objects that together cover the topic comprehensively from distinct angles.\n\n" + "Rules:\n" + "- Return exactly 3–7 sections\n" + "- Each section must cover a unique angle — no overlap between sections\n" + "- Titles must work as standalone note titles (specific, not generic like 'Overview')\n" + "- focus: one sentence describing exactly what this section covers\n" + "- Respond with ONLY a JSON array, no other text\n\n" + 'Example: [{"title": "CRISPR: Molecular Mechanisms", "focus": "How Cas9 identifies and cuts DNA at guide-RNA-specified sites"}]' + ), + }, + { + "role": "user", + "content": f"Topic: {topic}\n\nSources:\n{sources_block}", + }, + ] + try: + raw = await generate_completion(messages, model, max_tokens=400) + raw = raw.strip() + raw = re.sub(r"^```(?:json)?\s*", "", raw) + raw = re.sub(r"\s*```$", "", raw) + idx = raw.find("[") + if idx >= 0: + parsed, _ = _json.JSONDecoder().raw_decode(raw[idx:]) + if isinstance(parsed, list): + sections = [ + s for s in parsed + if isinstance(s, dict) and s.get("title") and s.get("focus") + ] + if len(sections) >= 3: + return sections[:8] + except Exception: + logger.warning("Outline generation failed for topic '%s'", topic, exc_info=True) + return [] + + +async def _synthesize_section( + section_title: str, + section_focus: str, + sources: list[dict], + model: str, +) -> tuple[str, str]: + """Synthesize one focused note section. + + Returns (section_title, body_markdown). Does not stream. + """ + sources_block = _build_sources_block(sources) if sources else "(no sources provided)" + messages = [ + { + "role": "system", + "content": ( + "You are a focused research writer. Write a single well-structured note section " + "on the specific topic provided.\n\n" + "Requirements:\n" + f"- Focus strictly on: {section_focus}\n" + "- 300–600 words of substantive prose\n" + "- Use ### for subsections only when they genuinely aid clarity\n" + "- Do NOT include a top-level # heading — the title is set separately\n" + "- Write in detailed prose paragraphs — not bullet points\n" + "- End with a '## Sources' section listing relevant source URLs as markdown hyperlinks\n" + "- Ignore source material that falls outside your assigned focus" + ), + }, + { + "role": "user", + "content": ( + f"Section title: {section_title}\n" + f"Focus: {section_focus}\n\n" + f"Sources:\n{sources_block}" + ), + }, + ] + raw = await generate_completion(messages, model, max_tokens=2048, num_ctx=16384) + return section_title, raw.strip() + + async def run_research_pipeline( topic: str, user_id: int, @@ -28,14 +128,17 @@ async def run_research_pipeline( buf=None, project_id: int | None = None, ) -> Note: - """Full research pipeline: search → fetch → synthesize → create note. + """Full research pipeline: search → fetch → outline → section notes → index note. - Emits status events via buf.append_event throughout (when buf is provided). - Returns the created Note. + Emits status events via buf throughout (when buf is provided). + Returns the index note (or a single fallback note on outline failure). """ + def _status(msg: str) -> None: + if buf is not None: + buf.append_event("status", {"status": msg}) + # Step 1: Generate sub-queries - if buf is not None: - buf.append_event("status", {"status": "Generating search queries..."}) + _status("Generating search queries...") queries = await _generate_sub_queries(topic, model) logger.info("Research: generated %d sub-queries for topic '%s'", len(queries), topic) @@ -43,8 +146,7 @@ async def run_research_pipeline( async def _search_with_stagger(i: int, query: str) -> tuple[str, list[dict]]: if i > 0: await asyncio.sleep(0.2 * i) - if buf is not None: - buf.append_event("status", {"status": f"Searching: {query}..."}) + _status(f"Searching: {query}...") results = await _search_searxng(query) logger.info("Research: query '%s' → %d results", query, len(results)) return query, results @@ -66,8 +168,7 @@ async def run_research_pipeline( # Fetch all unique URLs in parallel async def _fetch_source(url: str, result: dict, query: str) -> dict: title = result.get("title", url) - if buf is not None: - buf.append_event("status", {"status": f"Reading: {title[:60]}..."}) + _status(f"Reading: {title[:60]}...") content = await fetch_url_content(url) return { "url": url, @@ -84,39 +185,97 @@ async def run_research_pipeline( if not all_sources: raise ValueError(f"No results found for '{topic}'") - # Step 3: Filter failed fetches - good_sources = [ - s for s in all_sources - if not s["content"].startswith("[Failed to fetch") - ] + good_sources = [s for s in all_sources if not s["content"].startswith("[Failed to fetch")] if not good_sources: raise ValueError(f"Could not read any sources for '{topic}'") - # Limit to top N sources for synthesis (already deduplicated by URL) synthesis_sources = good_sources[:MAX_SYNTHESIS_SOURCES] logger.info( "Research: %d/%d sources successfully fetched, using %d for synthesis", len(good_sources), len(all_sources), len(synthesis_sources), ) - # Step 4: Synthesize (streams tokens into chat as the note is being written) - if buf is not None: - buf.append_event("status", {"status": f"Synthesizing report from {len(synthesis_sources)} sources..."}) - title, body = await _synthesize_note(topic, synthesis_sources, model, buf) + # Step 3: Generate topic outline + _status("Generating outline...") + outline = await _generate_outline(topic, synthesis_sources, model) - # Step 5: Create note - if buf is not None: - buf.append_event("status", {"status": "Saving note..."}) - note = await create_note( + # Fallback: outline failed or too short → single monolithic note + if not outline: + logger.warning("Research outline empty, falling back to single note for '%s'", topic) + _status("Synthesizing report...") + title, body = await _synthesize_note(topic, synthesis_sources, model, buf=None) + note = await create_note( + user_id=user_id, title=title, body=body, tags=["research"], project_id=project_id, + ) + logger.info("Research (fallback): created note id=%d title='%s'", note.id, note.title) + return note + + # Step 4: Synthesize each section in parallel + for section in outline: + _status(f"Writing: {section['title']}...") + + raw_results = await asyncio.gather( + *[_synthesize_section(s["title"], s["focus"], synthesis_sources, model) for s in outline], + return_exceptions=True, + ) + + # Step 5: Create section notes sequentially + _status(f"Saving {len(outline)} notes...") + section_note_pairs: list[tuple[dict, Note]] = [] + for section, result in zip(outline, raw_results): + if isinstance(result, Exception): + logger.warning("Section synthesis failed for '%s': %s", section["title"], result) + continue + sec_title, sec_body = result + try: + note = await create_note( + user_id=user_id, + title=sec_title, + body=sec_body, + tags=["research"], + project_id=project_id, + ) + section_note_pairs.append((section, note)) + except Exception: + logger.warning("Failed to save section note '%s'", sec_title, exc_info=True) + + # All sections failed — fall back to single note + if not section_note_pairs: + logger.warning("All section syntheses failed, falling back to single note for '%s'", topic) + _status("Synthesizing report (fallback)...") + title, body = await _synthesize_note(topic, synthesis_sources, model, buf=None) + note = await create_note( + user_id=user_id, title=title, body=body, tags=["research"], project_id=project_id, + ) + return note + + # Step 6: Create index note + from datetime import date as _date + index_lines = [ + f"Research overview for **{topic}** — {_date.today().isoformat()}", + "", + f"Generated from {len(synthesis_sources)} web sources across {len(section_note_pairs)} sections.", + "", + "## Sections", + "", + ] + for section, note in section_note_pairs: + index_lines.append(f"- **{note.title}** — {section['focus']}") + index_lines += ["", "*Search for any section title to read it.*"] + + index_note = await create_note( user_id=user_id, - title=title, - body=body, - tags=["research"], + title=f"Research: {topic}", + body="\n".join(index_lines), + tags=["research", "research-index"], project_id=project_id, ) - logger.info("Research: created note id=%d title='%s'", note.id, note.title) - return note + logger.info( + "Research: created %d section notes + index id=%d for topic '%s'", + len(section_note_pairs), index_note.id, topic, + ) + return index_note async def _generate_sub_queries(topic: str, model: str) -> list[str]: @@ -248,13 +407,7 @@ async def _synthesize_note( When buf is provided, tokens are streamed into the chat buffer in real time so the user can see the note being written. Uses an extended context window. """ - sources_text_parts = [] - for i, s in enumerate(sources, 1): - content = (s.get("content") or s.get("snippet") or "")[:CHARS_PER_SOURCE] - sources_text_parts.append( - f"[Source {i}] {s['title']}\nURL: {s['url']}\nSearch query: {s['query']}\n\n{content}" - ) - sources_block = "\n\n" + ("─" * 60) + "\n\n".join(sources_text_parts) + sources_block = _build_sources_block(sources) messages = [ { diff --git a/src/fabledassistant/services/tools.py b/src/fabledassistant/services/tools.py index dd58ba2..6b48abb 100644 --- a/src/fabledassistant/services/tools.py +++ b/src/fabledassistant/services/tools.py @@ -258,8 +258,8 @@ _CORE_TOOLS = [ "function": { "name": "get_note", "description": ( - "Retrieve the full content of a specific note. Use this when the user asks to read, " - "view, or check what a particular note says. Returns the complete note body." + "Retrieve the full content of a specific note or task. Use this when the user asks to read, " + "view, or check what a particular note or task says. Returns the complete body." ), "parameters": { "type": "object", @@ -316,7 +316,7 @@ _CORE_TOOLS = [ "name": "delete_note", "description": ( "Delete a note permanently. Use ONLY when the user explicitly asks to delete or remove a note. " - "This action requires user confirmation and cannot be undone." + "Always confirm with the user first — this cannot be undone." ), "parameters": { "type": "object", @@ -325,6 +325,10 @@ _CORE_TOOLS = [ "type": "string", "description": "Title or keyword to find the note to delete", }, + "confirmed": { + "type": "boolean", + "description": "Must be true — only set after the user has explicitly confirmed they want this note deleted.", + }, }, "required": ["query"], }, @@ -336,7 +340,7 @@ _CORE_TOOLS = [ "name": "delete_task", "description": ( "Delete a task permanently. Use ONLY when the user explicitly asks to delete or remove a task. " - "This action requires user confirmation and cannot be undone." + "Always confirm with the user first — this cannot be undone." ), "parameters": { "type": "object", @@ -345,6 +349,10 @@ _CORE_TOOLS = [ "type": "string", "description": "Title or keyword to find the task to delete", }, + "confirmed": { + "type": "boolean", + "description": "Must be true — only set after the user has explicitly confirmed they want this task deleted.", + }, }, "required": ["query"], }, @@ -937,6 +945,28 @@ _RAG_TOOLS = [ }, }, }, + { + "type": "function", + "function": { + "name": "calculate", + "description": ( + "Evaluate a mathematical expression and return the exact result. Use this for any " + "arithmetic, percentages, unit conversions, or multi-step calculations where precision " + "matters. Supports standard math operators (+, -, *, /, **, %) and all Python math " + "module functions (sqrt, log, sin, cos, floor, ceil, etc.)." + ), + "parameters": { + "type": "object", + "properties": { + "expression": { + "type": "string", + "description": "A valid Python math expression (e.g. '(450 * 1.13) / 12', 'sqrt(144)', 'log(1000, 10)')", + }, + }, + "required": ["expression"], + }, + }, + }, ] _ENTITY_TOOLS = [ @@ -964,6 +994,29 @@ _ENTITY_TOOLS = [ }, }, }, + { + "type": "function", + "function": { + "name": "update_person", + "description": ( + "Update details about a saved person. Use this when the user corrects or adds new " + "information about someone already in the knowledge base." + ), + "parameters": { + "type": "object", + "properties": { + "query": {"type": "string", "description": "Name or keyword to find the person"}, + "relationship": {"type": "string", "description": "Updated relationship to the user"}, + "phone": {"type": "string", "description": "Updated phone number"}, + "email": {"type": "string", "description": "Updated email address"}, + "birthday": {"type": "string", "description": "Updated birthday in YYYY-MM-DD format"}, + "address": {"type": "string", "description": "Updated home or mailing address"}, + "notes": {"type": "string", "description": "Updated free-form notes about this person"}, + }, + "required": ["query"], + }, + }, + }, { "type": "function", "function": { @@ -987,6 +1040,28 @@ _ENTITY_TOOLS = [ }, }, }, + { + "type": "function", + "function": { + "name": "update_place", + "description": ( + "Update details about a saved place. Use this when the user corrects or adds new " + "information about a location already in the knowledge base." + ), + "parameters": { + "type": "object", + "properties": { + "query": {"type": "string", "description": "Name or keyword to find the place"}, + "address": {"type": "string", "description": "Updated street address"}, + "phone": {"type": "string", "description": "Updated phone number"}, + "hours": {"type": "string", "description": "Updated opening hours"}, + "url": {"type": "string", "description": "Updated website URL"}, + "notes": {"type": "string", "description": "Updated free-form notes about this place"}, + }, + "required": ["query"], + }, + }, + }, { "type": "function", "function": { @@ -1051,12 +1126,71 @@ _ENTITY_TOOLS = [ ] +_PROFILE_TOOLS = [ + { + "type": "function", + "function": { + "name": "get_profile", + "description": ( + "Retrieve the user's stored profile: name, job title, industry, expertise level, " + "preferred response style, tone, and interests. Use this when you need to personalise " + "a response and the user's profile hasn't already been injected into context." + ), + "parameters": { + "type": "object", + "properties": {}, + }, + }, + }, + { + "type": "function", + "function": { + "name": "update_profile", + "description": ( + "Update the user's stored profile when they share personal information: their name, " + "job, industry, expertise, preferred response style, tone, or interests. " + "Only set fields the user has explicitly mentioned." + ), + "parameters": { + "type": "object", + "properties": { + "display_name": {"type": "string", "description": "User's preferred display name"}, + "job_title": {"type": "string", "description": "Job title (e.g. 'Software Engineer')"}, + "industry": {"type": "string", "description": "Industry (e.g. 'Healthcare', 'Finance')"}, + "expertise_level": { + "type": "string", + "enum": ["novice", "intermediate", "expert"], + "description": "User's general expertise level", + }, + "response_style": { + "type": "string", + "enum": ["concise", "balanced", "detailed"], + "description": "Preferred response length/depth", + }, + "tone": { + "type": "string", + "enum": ["casual", "professional", "technical"], + "description": "Preferred communication tone", + }, + "interests": { + "type": "array", + "items": {"type": "string"}, + "description": "List of topics or hobbies the user is interested in", + }, + }, + }, + }, + }, +] + + async def get_tools_for_user(user_id: int) -> list[dict]: """Build the tool list for a user based on their configured integrations.""" tools = list(_CORE_TOOLS) tools.extend(_URL_TOOLS) tools.extend(_RAG_TOOLS) tools.extend(_ENTITY_TOOLS) + tools.extend(_PROFILE_TOOLS) if await is_caldav_configured(user_id): tools.extend(_CALDAV_TOOLS) if Config.searxng_enabled(): @@ -1698,6 +1832,12 @@ async def execute_tool( note = notes[0] if note.status is not None: return {"success": False, "error": f"'{note.title}' is a task. Use delete_task instead."} + if not arguments.get("confirmed"): + return { + "success": False, + "requires_confirmation": True, + "error": f"Deleting '{note.title}' is permanent. Ask the user to confirm, then retry with confirmed=true.", + } deleted = await delete_note(user_id, note.id) if not deleted: return {"success": False, "error": "Failed to delete note."} @@ -1717,6 +1857,12 @@ async def execute_tool( note = notes[0] if note.status is None: return {"success": False, "error": f"'{note.title}' is a note. Use delete_note instead."} + if not arguments.get("confirmed"): + return { + "success": False, + "requires_confirmation": True, + "error": f"Deleting '{note.title}' is permanent. Ask the user to confirm, then retry with confirmed=true.", + } deleted = await delete_note(user_id, note.id) if not deleted: return {"success": False, "error": "Failed to delete task."} @@ -2172,6 +2318,72 @@ async def execute_tool( _schedule_embedding(note.id, user_id, name, note.body) return {"success": True, "type": "place", "data": {"id": note.id, "name": name}} + elif tool_name == "update_person": + query = str(arguments.get("query", "")).strip() + if not query: + return {"success": False, "error": "query is required"} + existing, _ = await list_notes(user_id=user_id, q=query, is_task=False, limit=10) + target = next((n for n in existing if n.note_type == "person"), None) + if target is None: + return {"success": False, "error": f"No person found matching '{query}'. Use create_person to add them."} + meta = dict(target.entity_meta or {}) + for field in ("relationship", "phone", "email", "birthday", "address"): + val = arguments.get(field) + if val is not None: + meta[field] = str(val) + body_lines = [] + if meta.get("relationship"): + body_lines.append(f"**Relationship:** {meta['relationship']}") + if meta.get("phone"): + body_lines.append(f"**Phone:** {meta['phone']}") + if meta.get("email"): + body_lines.append(f"**Email:** {meta['email']}") + if meta.get("birthday"): + body_lines.append(f"**Birthday:** {meta['birthday']}") + if meta.get("address"): + body_lines.append(f"**Address:** {meta['address']}") + extra_notes = arguments.get("notes") + if extra_notes is not None: + body_lines.append(f"\n{extra_notes}") + new_body = "\n".join(body_lines) + updated = await update_note(user_id=user_id, note_id=target.id, body=new_body, entity_meta=meta) + if updated is None: + return {"success": False, "error": "Failed to update person."} + _schedule_embedding(target.id, user_id, target.title, new_body) + return {"success": True, "type": "person_updated", "data": {"id": target.id, "name": target.title}} + + elif tool_name == "update_place": + query = str(arguments.get("query", "")).strip() + if not query: + return {"success": False, "error": "query is required"} + existing, _ = await list_notes(user_id=user_id, q=query, is_task=False, limit=10) + target = next((n for n in existing if n.note_type == "place"), None) + if target is None: + return {"success": False, "error": f"No place found matching '{query}'. Use create_place to add it."} + meta = dict(target.entity_meta or {}) + for field in ("address", "phone", "hours", "url"): + val = arguments.get(field) + if val is not None: + meta[field] = str(val) + body_lines = [] + if meta.get("address"): + body_lines.append(f"**Address:** {meta['address']}") + if meta.get("phone"): + body_lines.append(f"**Phone:** {meta['phone']}") + if meta.get("hours"): + body_lines.append(f"**Hours:** {meta['hours']}") + if meta.get("url"): + body_lines.append(f"**Website:** {meta['url']}") + extra_notes = arguments.get("notes") + if extra_notes is not None: + body_lines.append(f"\n{extra_notes}") + new_body = "\n".join(body_lines) + updated = await update_note(user_id=user_id, note_id=target.id, body=new_body, entity_meta=meta) + if updated is None: + return {"success": False, "error": "Failed to update place."} + _schedule_embedding(target.id, user_id, target.title, new_body) + return {"success": True, "type": "place_updated", "data": {"id": target.id, "name": target.title}} + elif tool_name == "create_list": name = str(arguments.get("name", "")).strip() if not name: @@ -2235,6 +2447,65 @@ async def execute_tool( _schedule_embedding(target.id, user_id, target.title, cleaned) return {"success": True, "type": "list_cleared", "data": {"id": target.id, "name": target.title}} + elif tool_name == "get_profile": + from fabledassistant.services.user_profile import get_profile as _get_profile + profile = await _get_profile(user_id) + return { + "success": True, + "type": "profile", + "data": { + "display_name": profile.display_name or "", + "job_title": profile.job_title or "", + "industry": profile.industry or "", + "expertise_level": profile.expertise_level or "intermediate", + "response_style": profile.response_style or "balanced", + "tone": profile.tone or "casual", + "interests": profile.interests or [], + }, + } + + elif tool_name == "update_profile": + from fabledassistant.services.user_profile import update_profile as _update_profile, VALID_EXPERTISE, VALID_STYLES, VALID_TONES + data: dict = {} + for field in ("display_name", "job_title", "industry"): + val = arguments.get(field) + if val is not None: + data[field] = str(val) + expertise = arguments.get("expertise_level") + if expertise in VALID_EXPERTISE: + data["expertise_level"] = expertise + style = arguments.get("response_style") + if style in VALID_STYLES: + data["response_style"] = style + tone = arguments.get("tone") + if tone in VALID_TONES: + data["tone"] = tone + interests = arguments.get("interests") + if isinstance(interests, list): + data["interests"] = [str(i) for i in interests if str(i).strip()] + if not data: + return {"success": False, "error": "No valid fields provided to update."} + profile = await _update_profile(user_id, data) + return { + "success": True, + "type": "profile_updated", + "data": {"fields_updated": list(data.keys())}, + } + + elif tool_name == "calculate": + import math as _math + expr = str(arguments.get("expression", "")).strip() + if not expr: + return {"success": False, "error": "expression is required"} + allowed_names = {k: v for k, v in vars(_math).items() if not k.startswith("_")} + allowed_names["abs"] = abs + allowed_names["round"] = round + try: + result = eval(expr, {"__builtins__": {}}, allowed_names) # noqa: S307 + except Exception as calc_err: + return {"success": False, "error": f"Could not evaluate expression: {calc_err}"} + return {"success": True, "type": "calculation", "data": {"expression": expr, "result": result}} + else: return {"success": False, "error": f"Unknown tool: {tool_name}"} diff --git a/src/fabledassistant/services/tts.py b/src/fabledassistant/services/tts.py index fa68d74..34f5263 100644 --- a/src/fabledassistant/services/tts.py +++ b/src/fabledassistant/services/tts.py @@ -239,17 +239,32 @@ async def synthesise( voice_param = _build_voice_param() t0 = time.monotonic() audio_chunks: list = [] - for _, _, audio in _pipeline(text, voice=voice_param, speed=speed): # type: ignore[misc] - if audio is not None: - audio_chunks.append(audio) + try: + for _, _, audio in _pipeline(text, voice=voice_param, speed=speed): # type: ignore[misc] + if audio is not None: + audio_chunks.append(audio) + except Exception: + logger.exception( + "Kokoro pipeline error during synthesis: %d chars, preview=%r", + len(text), text[:80], + ) + raise if not audio_chunks: + logger.warning( + "Kokoro produced no audio chunks: %d chars, preview=%r", + len(text), text[:80], + ) return b"" combined = np.concatenate(audio_chunks) buf = io.BytesIO() sf.write(buf, combined, samplerate=24000, format="WAV", subtype="PCM_16") - logger.debug("TTS synthesis took %.2fs for %d chars", time.monotonic() - t0, len(text)) + elapsed = time.monotonic() - t0 + logger.info( + "Kokoro synthesis: %d chars → %d samples (%.2fs, %.0f chars/s)", + len(text), len(combined), elapsed, len(text) / elapsed if elapsed > 0 else 0, + ) return buf.getvalue() loop = asyncio.get_running_loop() diff --git a/tests/test_research_pipeline.py b/tests/test_research_pipeline.py new file mode 100644 index 0000000..ae29516 --- /dev/null +++ b/tests/test_research_pipeline.py @@ -0,0 +1,186 @@ +"""Tests for the multi-note research pipeline.""" +import json +import pytest +from unittest.mock import AsyncMock, patch + + +@pytest.mark.asyncio +async def test_generate_outline_parses_valid_json(): + """_generate_outline returns parsed sections when model returns valid JSON.""" + from fabledassistant.services.research import _generate_outline + + outline_json = json.dumps([ + {"title": "Section One", "focus": "Focus one"}, + {"title": "Section Two", "focus": "Focus two"}, + {"title": "Section Three", "focus": "Focus three"}, + ]) + with patch("fabledassistant.services.research.generate_completion", new_callable=AsyncMock, return_value=outline_json): + result = await _generate_outline("test topic", [], "test-model") + + assert len(result) == 3 + assert result[0]["title"] == "Section One" + assert result[0]["focus"] == "Focus one" + + +@pytest.mark.asyncio +async def test_generate_outline_returns_empty_on_parse_failure(): + """_generate_outline returns [] when model output is not parseable JSON.""" + from fabledassistant.services.research import _generate_outline + + with patch("fabledassistant.services.research.generate_completion", new_callable=AsyncMock, return_value="not json at all"): + result = await _generate_outline("test topic", [], "test-model") + + assert result == [] + + +@pytest.mark.asyncio +async def test_generate_outline_returns_empty_when_fewer_than_3_sections(): + """_generate_outline returns [] when model returns fewer than 3 valid sections.""" + from fabledassistant.services.research import _generate_outline + + outline_json = json.dumps([{"title": "Only One", "focus": "Focus"}]) + with patch("fabledassistant.services.research.generate_completion", new_callable=AsyncMock, return_value=outline_json): + result = await _generate_outline("test topic", [], "test-model") + + assert result == [] + + +@pytest.mark.asyncio +async def test_generate_outline_truncates_to_8(): + """_generate_outline truncates results to at most 8 sections.""" + from fabledassistant.services.research import _generate_outline + + sections = [{"title": f"Section {i}", "focus": f"Focus {i}"} for i in range(10)] + with patch("fabledassistant.services.research.generate_completion", new_callable=AsyncMock, return_value=json.dumps(sections)): + result = await _generate_outline("test topic", [], "test-model") + + assert len(result) == 8 + + +@pytest.mark.asyncio +async def test_generate_outline_skips_malformed_entries(): + """_generate_outline filters out entries missing title or focus.""" + from fabledassistant.services.research import _generate_outline + + outline_json = json.dumps([ + {"title": "Good One", "focus": "Good focus"}, + {"title": "Missing focus"}, + {"focus": "Missing title"}, + {"title": "Good Two", "focus": "Good focus two"}, + {"title": "Good Three", "focus": "Good focus three"}, + ]) + with patch("fabledassistant.services.research.generate_completion", new_callable=AsyncMock, return_value=outline_json): + result = await _generate_outline("test topic", [], "test-model") + + assert len(result) == 3 + assert all(s["title"] and s["focus"] for s in result) + + +@pytest.mark.asyncio +async def test_synthesize_section_returns_title_and_body(): + """_synthesize_section returns (section_title, body) from model output.""" + from fabledassistant.services.research import _synthesize_section + + with patch("fabledassistant.services.research.generate_completion", new_callable=AsyncMock, return_value="The body of this section."): + title, body = await _synthesize_section( + section_title="Quantum Entanglement: Mechanisms", + section_focus="How entanglement works at the particle level", + sources=[], + model="test-model", + ) + + assert title == "Quantum Entanglement: Mechanisms" + assert body == "The body of this section." + + +@pytest.mark.asyncio +async def test_synthesize_section_strips_whitespace(): + """_synthesize_section strips leading/trailing whitespace from body.""" + from fabledassistant.services.research import _synthesize_section + + with patch("fabledassistant.services.research.generate_completion", new_callable=AsyncMock, return_value="\n\n body text \n\n"): + title, body = await _synthesize_section("Title", "Focus", [], "model") + + assert body == "body text" + + +@pytest.mark.asyncio +async def test_pipeline_creates_section_notes_and_index(): + """run_research_pipeline creates N section notes + 1 index note, returns index.""" + from unittest.mock import MagicMock + + outline = [ + {"title": "Section A", "focus": "Focus A"}, + {"title": "Section B", "focus": "Focus B"}, + {"title": "Section C", "focus": "Focus C"}, + ] + + note_id_counter = iter(range(10, 20)) + + def _make_note(user_id, title, body, tags, project_id=None): + n = MagicMock() + n.id = next(note_id_counter) + n.title = title + return n + + with patch("fabledassistant.services.research._generate_sub_queries", new_callable=AsyncMock, return_value=["q1"]), \ + patch("fabledassistant.services.research._search_searxng", new_callable=AsyncMock, return_value=[{"url": "http://x.com", "title": "X", "snippet": "s"}]), \ + patch("fabledassistant.services.research.fetch_url_content", new_callable=AsyncMock, return_value="content"), \ + patch("fabledassistant.services.research._generate_outline", new_callable=AsyncMock, return_value=outline), \ + patch("fabledassistant.services.research._synthesize_section", new_callable=AsyncMock, side_effect=lambda t, f, s, m: (t, f"Body for {t}")), \ + patch("fabledassistant.services.research.create_note", new_callable=AsyncMock, side_effect=_make_note): + + from fabledassistant.services.research import run_research_pipeline + result = await run_research_pipeline("test topic", user_id=1, model="test-model") + + assert result.title == "Research: test topic" + + +@pytest.mark.asyncio +async def test_pipeline_falls_back_to_single_note_when_outline_empty(): + """run_research_pipeline falls back to single-note synthesis when _generate_outline returns [].""" + from unittest.mock import MagicMock + + single_note = MagicMock() + single_note.id = 99 + single_note.title = "Research: test topic" + + with patch("fabledassistant.services.research._generate_sub_queries", new_callable=AsyncMock, return_value=["q1"]), \ + patch("fabledassistant.services.research._search_searxng", new_callable=AsyncMock, return_value=[{"url": "http://x.com", "title": "X", "snippet": "s"}]), \ + patch("fabledassistant.services.research.fetch_url_content", new_callable=AsyncMock, return_value="content"), \ + patch("fabledassistant.services.research._generate_outline", new_callable=AsyncMock, return_value=[]), \ + patch("fabledassistant.services.research._synthesize_note", new_callable=AsyncMock, return_value=("Research: test topic", "body")), \ + patch("fabledassistant.services.research.create_note", new_callable=AsyncMock, return_value=single_note): + + from fabledassistant.services.research import run_research_pipeline + result = await run_research_pipeline("test topic", user_id=1, model="test-model") + + assert result.id == 99 + + +@pytest.mark.asyncio +async def test_pipeline_falls_back_when_all_sections_fail(): + """run_research_pipeline falls back to single note when all section syntheses raise.""" + from unittest.mock import MagicMock + + outline = [ + {"title": "Section A", "focus": "Focus A"}, + {"title": "Section B", "focus": "Focus B"}, + {"title": "Section C", "focus": "Focus C"}, + ] + fallback_note = MagicMock() + fallback_note.id = 77 + fallback_note.title = "Research: test topic" + + with patch("fabledassistant.services.research._generate_sub_queries", new_callable=AsyncMock, return_value=["q1"]), \ + patch("fabledassistant.services.research._search_searxng", new_callable=AsyncMock, return_value=[{"url": "http://x.com", "title": "X", "snippet": "s"}]), \ + patch("fabledassistant.services.research.fetch_url_content", new_callable=AsyncMock, return_value="content"), \ + patch("fabledassistant.services.research._generate_outline", new_callable=AsyncMock, return_value=outline), \ + patch("fabledassistant.services.research._synthesize_section", new_callable=AsyncMock, side_effect=RuntimeError("synthesis failed")), \ + patch("fabledassistant.services.research._synthesize_note", new_callable=AsyncMock, return_value=("Research: test topic", "body")), \ + patch("fabledassistant.services.research.create_note", new_callable=AsyncMock, return_value=fallback_note): + + from fabledassistant.services.research import run_research_pipeline + result = await run_research_pipeline("test topic", user_id=1, model="test-model") + + assert result.id == 77