Commit Graph

19 Commits

Author SHA1 Message Date
bvandeusen 53e54ea761 Remove intent router from chat pipeline; raise OLLAMA_NUM_CTX to 16384
The intent classifier (Phase 21) is removed from the main chat generation
path. The main model now handles all tool routing natively via Ollama's
structured tool-calling API, eliminating misidentification issues caused
by the small intent model.

Changes:
- generation_task.py: remove classify_intent call, intent_task, _WRITE_TOOLS,
  _TOOL_ACTIONS, _INTENT_TRIGGER_WORDS, _should_skip_intent(), and the entire
  round-0 intent-first + write-tool confirmation block (~315 lines removed)
- research_topic tool calls are now handled inline in the streaming loop:
  runs run_research_pipeline, streams synthesis to buf, then breaks the round
  loop (research is still the full response, no model follow-up)
- config.py: raise OLLAMA_NUM_CTX default from 8192 to 16384

The quick-capture dedicated classifier (classify_capture_intent) is unchanged.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-02 18:30:21 -05:00
bvandeusen efe0a15b8c Add image search with local cache (Phase 23)
Images found via SearXNG are fetched server-side, stored on disk, and
served from /api/images/<id> — the user's browser never contacts the
original image host.  Original URLs are preserved for citation.

New files:
- alembic/versions/0016_add_image_cache.py  — image_cache table
- src/fabledassistant/models/image_cache.py — SQLAlchemy model
- src/fabledassistant/services/images.py    — fetch/store/serve logic
- src/fabledassistant/routes/images.py      — GET /api/images/<id>

Modified:
- config.py: IMAGE_CACHE_DIR (/data/images), IMAGE_MAX_BYTES (5 MB)
- research.py: _search_searxng_images() — SearXNG categories=images
- tools.py: _IMAGE_TOOLS def + search_images branch in execute_tool
- intent.py: search_images routing rule (explicit visual language only)
- app.py: register images_bp
- docker-compose.yml: image_cache named volume mounted at /data/images
- ToolCallCard.vue: "image_search" label

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-01 12:55:10 -05:00
bvandeusen 926e4bb570 Upgrade intent model to qwen2.5:7b, simplify intent prompt rules
config.py:
- Default OLLAMA_INTENT_MODEL: qwen2.5:1.5b → qwen2.5:7b
- Startup will auto-pull and warm the new model on next container restart

intent.py:
- Replaced phrase-matching examples in search_web and research_topic rules
  with semantic descriptions. The 7B model doesn't need example phrases to
  understand intent — it can reason from the tool's purpose. Removes implied
  usage patterns that caused misclassifications on conversational phrasing
  (e.g. "I've been thinking about buying shirts, can you research this?").
- research_topic rule now explicitly covers any subject regardless of phrasing,
  including shopping decisions, comparisons, how-things-work questions, etc.
- search_web rule clarified as "short summary, no note" vs research_topic's
  "comprehensive written reference"

The 1.5B model required prescriptive phrase examples to route correctly; the
7B model has sufficient language understanding to classify from semantic intent.
Expected improvement: ~1-2s intent calls (vs 0.4-9s for the 1.5B model which
sometimes timed out or misclassified longer/conversational messages).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-28 15:51:27 -05:00
bvandeusen 590682a5d2 Phase 22: SearXNG web research pipeline + settings layout overhaul
Research pipeline (research_topic tool):
- New service: services/research.py — sub-query generation, SearXNG
  search, URL fetch, deduplication, and LLM synthesis into a note
- 5 sub-queries × 3 pages = up to 15 sources, capped at 12 for synthesis
- Synthesis uses num_ctx=16384 + max_tokens=8192 for long-form output
- Prompt demands 2500+ words, 6+ topic-appropriate sections, detailed prose
- 429 retry with backoff; 1s inter-query sleep; raw_decode JSON parsing

search_web tool (new):
- Lightweight single-query SearXNG search, results returned inline in chat
- LLM answers conversationally in round 1; no note created
- web_search result type with external links in ToolCallCard

Infrastructure:
- llm.py: generate_completion accepts num_ctx override
- config.py: SEARXNG_URL + Config.searxng_enabled()
- docker-compose: OLLAMA_NUM_PARALLEL=2, commented SEARXNG_URL example
- intent.py: search_web and research_topic routing rules

Settings UI:
- 2-column grid layout (small sections pair up, complex span full width)
- Search Test section: live SearXNG query with result preview
- GET /api/settings/search?q= proxy endpoint
- Research button (magnifier) in ChatView input toolbar → popover modal

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-27 15:21:38 -05:00
bvandeusen 8ee3649531 Use qwen3:latest as default model to avoid tag ambiguity
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-25 22:31:35 -05:00
bvandeusen f45b7cf9c5 Add model dropdowns to settings and set qwen2.5:1.5b as intent default
- Default OLLAMA_INTENT_MODEL to qwen2.5:1.5b in code instead of empty
- Add GET /api/settings/models endpoint returning installed models and defaults
- Validate intent_model against installed models on save (same as default_model)
- Replace intent model text input with a dropdown of installed models
- Add chat model dropdown to Assistant settings section

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-25 22:17:45 -05:00
bvandeusen 7b6248c8a7 Add OLLAMA_NUM_CTX config to reduce VRAM usage
Replaces the hardcoded num_ctx=32768 KV cache allocation with a
configurable env var defaulting to 8192. This significantly reduces
VRAM pressure when multiple services share the GPU.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-25 22:02:06 -05:00
bvandeusen b37e15d59a Add Authentik OAuth/OIDC SSO, email change, and setup docs
Phase 18 changes:

OAuth/OIDC SSO (Authorization Code + PKCE):
- alembic/versions/0015_add_oauth_fields.py: add oauth_sub UNIQUE column,
  drop NOT NULL on password_hash
- src/fabledassistant/services/oauth.py: OIDC discovery (cached), build_auth_url,
  exchange_code, get_userinfo, find_or_create_oauth_user (sub→email auto-link→create)
- src/fabledassistant/routes/auth.py: GET /api/auth/oauth/login and
  GET /api/auth/oauth/callback; LOCAL_AUTH_ENABLED guards on login/register;
  /api/auth/status now returns oauth_enabled + local_auth_enabled
- src/fabledassistant/config.py: OIDC_ISSUER, OIDC_CLIENT_ID, OIDC_CLIENT_SECRET,
  OIDC_SCOPES, LOCAL_AUTH_ENABLED, oidc_enabled() classmethod
- src/fabledassistant/models/user.py: password_hash nullable, oauth_sub field,
  has_password bool in to_dict()
- src/fabledassistant/services/auth.py: create_user accepts password=None +
  oauth_sub kwarg; authenticate returns None for OAuth-only users;
  add get_user_by_oauth_sub, link_oauth_sub, update_user_email
- frontend: AuthStatus + User types updated; auth store exposes oauthEnabled +
  localAuthEnabled; LoginView shows SSO button / hides password form accordingly

Email change:
- PUT /api/auth/email: requires password confirmation for local-auth users,
  skips check for OAuth-only users; enforces email uniqueness
- SettingsView.vue: new Email Address section pre-filled with current email,
  updates authStore.user in-place on success

Docs:
- docs/oauth-setup.md: step-by-step Authentik provider setup, example
  docker-compose env vars, account linking explanation, per-provider issuer URL table

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-25 20:12:13 -05:00
bvandeusen 667ccd70cd Security audit, bug fixes, auto-save, and writing assistant improvements
Backend security & correctness:
- Add rate_limit.py: sliding-window rate limiter (asyncio) applied to login,
  register, forgot/reset password endpoints (10/60s or 5/300s per IP)
- app.py: add security headers in after_request (X-Frame-Options, CSP,
  X-Content-Type-Options, Referrer-Policy) using setdefault to preserve SSE headers
- auth.py: refactor duplicate login_required/admin_required into shared _check_auth()
- config.py: add TRUST_PROXY_HEADERS for proxy-aware client IP resolution
- routes/auth.py: rate limiting, _client_ip() helper, cleaned-up reset_password route
- routes/chat.py, notes.py, tasks.py: int() DoS fix on last_event_id; limit capped
  at 500; date.fromisoformat() wrapped in try/except → 400 on invalid dates
- services/auth.py: fix Setting.user_id update bug (filter on NULL not user.id);
  reset_password_with_token returns int|None (user_id) instead of bool
- services/backup.py: add _security_notice to full backup JSON export
- services/assist.py: system prompt explicitly preserves markdown list structure
  and nested indented sub-items

Infrastructure:
- docker-compose.yml: add healthcheck on app service (/api/health, 10s interval)
- .dockerignore: prevent secrets/node_modules/__pycache__/.env.* leaking into build

Frontend bug fixes:
- TaskCard.vue, TaskViewerView.vue: fix isOverdue() timezone bug (ISO string compare)
- useAssist.ts: accept() now resets state to idle when document changed since proposal
- stores/chat.ts: fix memory leak in _pollUntilLoaded() (try/catch around fetchStatus)
- TiptapEditor.vue: selection offset uses closest-match strategy (not first-match)
- utils/markdown.ts: explicit DOMPurify config with FORCE_BODY; remove as const
  (DOMPurify expects mutable string[])

New features:
- Auto-save (5-minute interval) in NoteEditorView and TaskEditorView — only when
  editing an existing dirty record; silent on error, shows "Auto-saved" toast
- sectionParser.ts: top-level bullet/numbered list items are now individual sections
  in the AI Assist panel (previously treated as one undifferentiated block)
- editor-shared.css: extracted ~500 lines of CSS duplicated between both editors;
  includes .inline-assist-btn at global scope (required for teleported elements)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-19 20:08:44 -05:00
bvandeusen d6f4a6dbb6 Add semantic note search (nomic-embed-text) and per-conversation note cache
- New NoteEmbedding model + migration 0014 stores float embeddings (JSONB)
- services/embeddings.py: get_embedding, upsert_note_embedding,
  semantic_search_notes (cosine similarity), backfill_note_embeddings
- build_context() now tries semantic search first, falls back to keyword search;
  accepts cached_note_ids to reuse last-turn notes and stabilise the system
  prompt prefix for Ollama's KV cache
- generation_buffer.py: per-conversation note ID cache (get/set/clear)
- generation_task.py: passes cached IDs into build_context, updates cache
  after each turn, and invalidates it after create_note/update_note/create_task
- app.py: pulls nomic-embed-text at startup and launches a background backfill
  to embed all existing notes (30 s delay so Ollama has time to load the model)
- routes/notes.py + services/tools.py: fire-and-forget embedding update on
  every note create or update via the API or LLM tool calls

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-18 21:44:58 -05:00
bvandeusen 70cba72a80 Phase 10: CalDAV full lifecycle, update_note, dashboard inline streaming, keyboard shortcuts
Backend:
- caldav.py: Full event lifecycle — update_event, delete_event; VTODO suite —
  create_todo, list_todos, complete_todo, delete_todo; list_calendars; timezone
  support via ZoneInfo; reminders via VALARM; attendees; multi-calendar search
  (_get_all_calendars scans all calendars when no specific one is configured)
- tools.py: New update_note tool (find by title + replace/append modes),
  7 new CalDAV tool definitions, corresponding execute_tool cases
- llm.py: Update system prompt — add update_note guidance, full CalDAV action list
- intent.py: Confidence scoring (high/medium/low) + should_execute property;
  conversation history support for anaphora resolution; routing rules for
  update/delete events, todos, update_note vs create_note disambiguation,
  time-period → list_events (not search_events), reminder_minutes conversion
- generation_task.py: Parallel fetch of tools + intent_model setting; dedicated
  intent model (OLLAMA_INTENT_MODEL env var or per-user intent_model setting)
- config.py: Add OLLAMA_INTENT_MODEL env var

Frontend:
- HomeView.vue: Inline streaming response (no navigation); quick action chips;
  isConversational computed — prominent "Continue this conversation" CTA when
  no tool calls; auto-focus chat input on mount via chatInputRef
- DashboardChatInput.vue: defineExpose({ focus }) for external focus control
- ChatView.vue: Escape key handler — close picker → close sidebar → clear
  textarea → navigate home; onUnmounted cleanup
- App.vue: Global ? key shortcut toggles keyboard shortcuts overlay; shared
  state via useShortcuts composable; Transition animation
- AppHeader.vue: ? button for shortcuts overlay discoverability
- useShortcuts.ts (new): Shared showShortcuts ref + open/close/toggle helpers
- ToolCallCard.vue: note_updated, event_updated, event_deleted, calendars,
  todo, todos, todo_completed, todo_deleted label cases + render blocks
- SettingsView.vue: Intent model field + caldav_timezone setting

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-17 22:04:41 -05:00
bvandeusen 75560dee4e Switch default model to qwen3 and add intent routing for reliable tool calling
Mistral didn't reliably use Ollama's structured tool calling API — it wrote
tool calls as JSON text instead of invoking them. This adds an intent routing
layer that classifies user intent via a fast non-streaming LLM call before
streaming, executing detected tools directly and bypassing native tool calling.

- Change default OLLAMA_MODEL from mistral to qwen3
- Add intent.py: classify_intent() with JSON parsing and fallback regex
- Integrate intent routing into generation_task.py round 0
- Add all-day event support (iCalendar DATE values) to CalDAV service
- Add recurring event support (RRULE) to CalDAV service and tool definition
- Improve create_event tool description for descriptive titles
- Enhance system prompt with structured tool usage guidance

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-16 16:24:01 -05:00
bvandeusen d7bc3f3222 Add CalDAV calendar integration, LLM-suggested tags, and settings refinements
- CalDAV integration: per-user calendar config, create/list/search events
  via caldav library, LLM tools for calendar operations from chat
- LLM-suggested tags: new tag_suggestions service prompts LLM with existing
  tags and note content to suggest 3-5 relevant tags; exposed via API
  endpoints (suggest-tags, append-tag); integrated into editor views
  (suggest button + clickable pills) and chat tool calls (pills in
  ToolCallCard with one-click apply)
- Settings/model UI refinements, generation task improvements

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-15 22:40:20 -05:00
bvandeusen d354da5b51 Add email-based password reset flow
Users who forget their password can now request a reset link via email.
Tokens are SHA-256 hashed before storage, expire after 1 hour, and
previous unused tokens are invalidated on new requests. The forgot-password
endpoint always returns success to prevent email enumeration.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-13 21:42:45 -05:00
bvandeusen d874e0e2ae Add application logging, SMTP email notifications, and supporting changes
Phase 5.4 — Application Logging:
- AppLog model + migration 0010 for unified audit/usage/error logging
- Usage logging middleware in app.py (after_request for /api/* requests)
- Error logging in 500 handler with traceback capture
- Audit logging for auth events (register, login, login_failed, logout,
  password_change) and admin actions (backup, restore, user_delete,
  registration_toggle, smtp_config, smtp_test)
- Admin log viewer (LogsView.vue) with stats, category/search/date
  filters, paginated table with expandable detail rows
- Admin logs API endpoints in admin.py (GET /logs, GET /logs/stats)
- Configurable retention via LOG_RETENTION_DAYS with hourly cleanup

Phase 5.5 — SMTP Email Notifications:
- aiosmtplib dependency for async email sending
- Email service (services/email.py) with STARTTLS/implicit TLS support
- Notification service (services/notifications.py) for security alerts
  and task due date reminders with per-user preferences
- Admin SMTP config endpoints (GET/PUT /api/admin/smtp, POST test)
- SMTP config in Config class with env var + Docker secret support
- Settings UI: notification preferences for all users, SMTP config
  section for admin with test email

Other changes:
- stream_chat() now accepts optional options dict (for num_predict)
- Increase assist MAX_BODY_CHARS from 3000 to 8000
- get_user_by_username() added to auth service
- apiStreamPost buffer processing refactored for robustness
- AppHeader: admin Logs nav link
- Router: /admin/logs route

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-13 08:34:52 -05:00
bvandeusen f77b029943 Add registration control, admin user management, and security hardening
- Registration auto-closes after first user; admin can toggle from /admin/users
- Admin user management view with user list and delete
- Password confirmation on registration form
- Password change in Settings (PUT /api/auth/password)
- Session cookie hardening: HttpOnly, SameSite=Lax, optional Secure flag
- Startup warning when SECRET_KEY is default
- Production deployment docs: reverse proxy, rate limiting, CSP headers
- Fix assist prompt to preserve markdown headings in target sections
- Simplify prod compose networking

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-12 17:49:22 -05:00
bvandeusen cbfdf5289e Add multi-user auth, background generation, and chat UX improvements
Phase 5: Multi-user authentication with session cookies, bcrypt passwords,
first-user-is-admin pattern, per-user data isolation, backup/restore,
Docker Swarm production stack with secrets and network isolation.

Phase 5.1: Chat UX improvements:
- Background generation architecture (GenerationBuffer + asyncio task)
  with SSE fan-out, reconnect support, and periodic DB flushes
- LLM-generated conversation titles (first exchange + every 10th message)
- Stop generation button with cancel_event and partial content preservation
- Relative timestamps in sidebar (5m ago, 3h ago, then dates)
- Empty chat auto-cleanup on navigation away
- Save-as-note uses LLM for title generation, tags notes with "chat"
- Summarize-as-note also tags with "chat"

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-11 14:36:30 -05:00
bvandeusen d2b8ab8fe8 Add LLM chat integration with streaming responses via Ollama
Phase 4: Full chat system with SSE streaming, note-aware context, and
conversation persistence.

Backend:
- Migration 0005: conversations + messages tables with FKs and indexes
- Conversation/Message SQLAlchemy models with relationships
- LLM service: ensure_model (auto-pull on startup), stream_chat (NDJSON),
  generate_completion, fetch_url_content (HTML stripping), build_context
  (keyword extraction, related note search, URL content injection)
- Chat service: conversation CRUD, save_response_as_note,
  summarize_conversation_as_note
- Chat routes blueprint: 9 endpoints including SSE streaming for messages,
  save/summarize as note, Ollama model listing
- Auto-pull llama3.1 model on app startup (non-blocking)

Frontend:
- apiStreamPost: SSE client using fetch + ReadableStream
- Chat Pinia store with streaming state management
- ChatView: dedicated /chat page with conversation sidebar + message thread
- ChatPanel: slide-out panel with contextNoteId from current route
- ChatMessage: markdown-rendered message bubble with "Save as Note" action
- Updated AppHeader with Chat nav link + panel toggle button
- Updated App.vue to mount ChatPanel with route-derived context
- Added /chat and /chat/:id routes

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-10 18:45:22 -05:00
bvandeusen 22a3a3c1d1 Initial commit: note-taking/task-tracking app with LLM integration scaffold
Vue 3 + TypeScript frontend with Pinia stores, markdown rendering (marked + DOMPurify),
wikilink/tag linkification, and autocomplete. Quart async backend with SQLAlchemy 2.0,
PostgreSQL ARRAY columns, task-note companion linking, backlinks, and note-to-task
conversion. Docker Compose setup with PostgreSQL 16 and Ollama.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-09 23:35:44 -05:00