Commit Graph

16 Commits

Author SHA1 Message Date
bvandeusen 7d532b82a6 Fix startup warm-up to use user model settings instead of env var
Startup no longer auto-warms Config.OLLAMA_MODEL. Instead it queries
all distinct default_model values from user settings, cross-references
with Ollama's installed models, and warms only the intersection.

Models that users have selected but not yet installed are skipped with
an info log — they are never auto-pulled. The embedding model pull
behaviour is unchanged.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-02 22:58:20 -05:00
bvandeusen 012eb1d46b Add Projects, Milestones, RAG auto-inject, push notifications, PWA, tag normalisation
## Projects & Milestones (Phases A + G)
- New models: Project, Milestone (Project → Milestone → Task hierarchy)
- notes table: project_id + milestone_id FKs; parent_id FK constraint activated
- Migrations: 0017 (projects), 0018 (push_subscriptions), 0019 (events), 0020 (milestones)
- Services: projects.py, milestones.py (CRUD + progress tracking)
- Routes: /api/projects + /api/projects/<id>/milestones
- LLM tools: create/list/get/update project; create/list milestone; project + milestone + parent_task params on note/task tools
- Frontend: ProjectListView (stacked milestone bars), ProjectView (milestone-grouped kanban), ProjectSelector, MilestoneSelector, NoteEditorView + TaskEditorView updated

## RAG Auto-injection (Phase B)
- Notes ≥0.60 cosine similarity auto-injected into system prompt (max 3, 800 chars each)
- excluded_note_ids param; ChatView "Auto-included" sidebar section

## Summarisation improvements (Phase C)
- Threshold 20→30, keep-recent 6→8, max_tokens 200→400
- Two-pass summarisation for histories >50 messages

## Browser push notifications (Phase E)
- PushSubscription model + migration; pywebpush dependency
- /api/push routes; VAPID config; fire-and-forget on generation complete
- Frontend: sw.js, push store, Settings toggle

## PWA manifest (Phase F)
- manifest.json, Apple meta tags, service worker registration in main.ts

## Tag normalisation
- All tags lowercased + deduplicated at backend (create_note/update_note) and frontend (TagInput sanitize)
- Note/Task types gain project_id + milestone_id fields; store signatures updated

## CalDAV
- Radicale embedded server reverted; back to user-configured external CalDAV

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-02 20:52:21 -05:00
bvandeusen 3d7be5888e Remove intent model entirely; quick-capture uses primary model
The separate intent model (OLLAMA_INTENT_MODEL / qwen2.5:7b) is removed
from every part of the system. All classification now uses the primary model.

Changes:
- config.py: remove OLLAMA_INTENT_MODEL
- intent.py: remove classify_intent() and all supporting infrastructure
  (_SYSTEM_PROMPT_TEMPLATE, _RESEARCH_PREFIX, _PRIOR_WORK_REFS); file now
  only contains the quick-capture classifier
- quick_capture.py: classify_capture_intent() now called with Config.OLLAMA_MODEL
- generation_task.py: remove intent_model_setting DB lookup and get_setting import;
  history summarization and research pipeline use the primary model directly
- research.py: remove intent_model parameter from run_research_pipeline() and
  _generate_sub_queries(); both use the model param throughout
- routes/settings.py: remove intent_model from model-key validation and response
- app.py: remove intent model pre-warming at startup
- SettingsView.vue: remove Intent Model selector and related refs/state

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-02 18:41:49 -05:00
bvandeusen efe0a15b8c Add image search with local cache (Phase 23)
Images found via SearXNG are fetched server-side, stored on disk, and
served from /api/images/<id> — the user's browser never contacts the
original image host.  Original URLs are preserved for citation.

New files:
- alembic/versions/0016_add_image_cache.py  — image_cache table
- src/fabledassistant/models/image_cache.py — SQLAlchemy model
- src/fabledassistant/services/images.py    — fetch/store/serve logic
- src/fabledassistant/routes/images.py      — GET /api/images/<id>

Modified:
- config.py: IMAGE_CACHE_DIR (/data/images), IMAGE_MAX_BYTES (5 MB)
- research.py: _search_searxng_images() — SearXNG categories=images
- tools.py: _IMAGE_TOOLS def + search_images branch in execute_tool
- intent.py: search_images routing rule (explicit visual language only)
- app.py: register images_bp
- docker-compose.yml: image_cache named volume mounted at /data/images
- ToolCallCard.vue: "image_search" label

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-01 12:55:10 -05:00
bvandeusen b079c583df Add /api/quick-capture endpoint for mobile/external clients
Single POST that classifies natural-language text and creates the
appropriate item (note, task, event, or todo) in one synchronous
request — no SSE, no conversation context needed.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-28 21:56:16 -05:00
bvandeusen b23e78b0ac Warm both chat and intent models into VRAM on startup
ensure_model only downloads a model if missing; it does not load it
into VRAM. The frontend warm call only covers the chat model (and only
after a user opens the dashboard). This left qwen2.5:1.5b (intent) cold,
causing simultaneous cold-load 500s when the first chat arrived.

Now both Config.OLLAMA_MODEL and Config.OLLAMA_INTENT_MODEL are warmed
at startup (after ensuring they're installed) via a fire-and-forget
/api/generate call with keep_alive=30m. The embedding model is still
pulled but not warmed (it's loaded on demand during backfill).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-25 23:18:07 -05:00
bvandeusen 667ccd70cd Security audit, bug fixes, auto-save, and writing assistant improvements
Backend security & correctness:
- Add rate_limit.py: sliding-window rate limiter (asyncio) applied to login,
  register, forgot/reset password endpoints (10/60s or 5/300s per IP)
- app.py: add security headers in after_request (X-Frame-Options, CSP,
  X-Content-Type-Options, Referrer-Policy) using setdefault to preserve SSE headers
- auth.py: refactor duplicate login_required/admin_required into shared _check_auth()
- config.py: add TRUST_PROXY_HEADERS for proxy-aware client IP resolution
- routes/auth.py: rate limiting, _client_ip() helper, cleaned-up reset_password route
- routes/chat.py, notes.py, tasks.py: int() DoS fix on last_event_id; limit capped
  at 500; date.fromisoformat() wrapped in try/except → 400 on invalid dates
- services/auth.py: fix Setting.user_id update bug (filter on NULL not user.id);
  reset_password_with_token returns int|None (user_id) instead of bool
- services/backup.py: add _security_notice to full backup JSON export
- services/assist.py: system prompt explicitly preserves markdown list structure
  and nested indented sub-items

Infrastructure:
- docker-compose.yml: add healthcheck on app service (/api/health, 10s interval)
- .dockerignore: prevent secrets/node_modules/__pycache__/.env.* leaking into build

Frontend bug fixes:
- TaskCard.vue, TaskViewerView.vue: fix isOverdue() timezone bug (ISO string compare)
- useAssist.ts: accept() now resets state to idle when document changed since proposal
- stores/chat.ts: fix memory leak in _pollUntilLoaded() (try/catch around fetchStatus)
- TiptapEditor.vue: selection offset uses closest-match strategy (not first-match)
- utils/markdown.ts: explicit DOMPurify config with FORCE_BODY; remove as const
  (DOMPurify expects mutable string[])

New features:
- Auto-save (5-minute interval) in NoteEditorView and TaskEditorView — only when
  editing an existing dirty record; silent on error, shows "Auto-saved" toast
- sectionParser.ts: top-level bullet/numbered list items are now individual sections
  in the AI Assist panel (previously treated as one undifferentiated block)
- editor-shared.css: extracted ~500 lines of CSS duplicated between both editors;
  includes .inline-assist-btn at global scope (required for teleported elements)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-19 20:08:44 -05:00
bvandeusen d6f4a6dbb6 Add semantic note search (nomic-embed-text) and per-conversation note cache
- New NoteEmbedding model + migration 0014 stores float embeddings (JSONB)
- services/embeddings.py: get_embedding, upsert_note_embedding,
  semantic_search_notes (cosine similarity), backfill_note_embeddings
- build_context() now tries semantic search first, falls back to keyword search;
  accepts cached_note_ids to reuse last-turn notes and stabilise the system
  prompt prefix for Ollama's KV cache
- generation_buffer.py: per-conversation note ID cache (get/set/clear)
- generation_task.py: passes cached IDs into build_context, updates cache
  after each turn, and invalidates it after create_note/update_note/create_task
- app.py: pulls nomic-embed-text at startup and launches a background backfill
  to embed all existing notes (30 s delay so Ollama has time to load the model)
- routes/notes.py + services/tools.py: fire-and-forget embedding update on
  every note create or update via the API or LLM tool calls

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-18 21:44:58 -05:00
bvandeusen 504091bfcf Pull intent model on startup alongside main model
If OLLAMA_INTENT_MODEL is configured and differs from OLLAMA_MODEL,
both are pulled concurrently as fire-and-forget tasks at startup.
Deduplicates via a set so pulling the same model twice is never attempted.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-18 19:37:36 -05:00
bvandeusen d874e0e2ae Add application logging, SMTP email notifications, and supporting changes
Phase 5.4 — Application Logging:
- AppLog model + migration 0010 for unified audit/usage/error logging
- Usage logging middleware in app.py (after_request for /api/* requests)
- Error logging in 500 handler with traceback capture
- Audit logging for auth events (register, login, login_failed, logout,
  password_change) and admin actions (backup, restore, user_delete,
  registration_toggle, smtp_config, smtp_test)
- Admin log viewer (LogsView.vue) with stats, category/search/date
  filters, paginated table with expandable detail rows
- Admin logs API endpoints in admin.py (GET /logs, GET /logs/stats)
- Configurable retention via LOG_RETENTION_DAYS with hourly cleanup

Phase 5.5 — SMTP Email Notifications:
- aiosmtplib dependency for async email sending
- Email service (services/email.py) with STARTTLS/implicit TLS support
- Notification service (services/notifications.py) for security alerts
  and task due date reminders with per-user preferences
- Admin SMTP config endpoints (GET/PUT /api/admin/smtp, POST test)
- SMTP config in Config class with env var + Docker secret support
- Settings UI: notification preferences for all users, SMTP config
  section for admin with test email

Other changes:
- stream_chat() now accepts optional options dict (for num_predict)
- Increase assist MAX_BODY_CHARS from 3000 to 8000
- get_user_by_username() added to auth service
- apiStreamPost buffer processing refactored for robustness
- AppHeader: admin Logs nav link
- Router: /admin/logs route

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-13 08:34:52 -05:00
bvandeusen f77b029943 Add registration control, admin user management, and security hardening
- Registration auto-closes after first user; admin can toggle from /admin/users
- Admin user management view with user list and delete
- Password confirmation on registration form
- Password change in Settings (PUT /api/auth/password)
- Session cookie hardening: HttpOnly, SameSite=Lax, optional Secure flag
- Startup warning when SECRET_KEY is default
- Production deployment docs: reverse proxy, rate limiting, CSP headers
- Fix assist prompt to preserve markdown headings in target sections
- Simplify prod compose networking

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-12 17:49:22 -05:00
bvandeusen cbfdf5289e Add multi-user auth, background generation, and chat UX improvements
Phase 5: Multi-user authentication with session cookies, bcrypt passwords,
first-user-is-admin pattern, per-user data isolation, backup/restore,
Docker Swarm production stack with secrets and network isolation.

Phase 5.1: Chat UX improvements:
- Background generation architecture (GenerationBuffer + asyncio task)
  with SSE fan-out, reconnect support, and periodic DB flushes
- LLM-generated conversation titles (first exchange + every 10th message)
- Stop generation button with cancel_event and partial content preservation
- Relative timestamps in sidebar (5m ago, 3h ago, then dates)
- Empty chat auto-cleanup on navigation away
- Save-as-note uses LLM for title generation, tags notes with "chat"
- Summarize-as-note also tags with "chat"

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-11 14:36:30 -05:00
bvandeusen 38b1ac933e Add settings page, model management, and chat UX improvements
- Settings infrastructure: key-value settings table, GET/PUT API, Pinia store
- Configurable assistant name (default "Fable") in settings and LLM system prompt
- Model catalog with 18 models in 3 categories (General Purpose, Coding,
  Uncensored / Creative Writing) with download/select/remove functionality
- Move Ollama status indicator from chat views to global nav bar
- Chat bubble layout: user messages right-aligned, assistant left-aligned
- Floating dark input bar with auto-focus and circular send button
- Fix HTML entity rendering (&#39; apostrophe issue in marked/DOMPurify pipeline)
- Fix new chat button navigation (fetchConversation before router.push)
- Recent chats section on home page with "New Chat" button
- Update summary.md with Phase 4.5 changes

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-10 21:32:02 -05:00
bvandeusen f4c7deba66 Run model pull as background task to avoid startup timeout
ensure_model() can take minutes to pull a multi-GB model, which
exceeds Hypercorn's startup timeout. Now fires as an asyncio
background task so the app starts immediately.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-10 19:16:54 -05:00
bvandeusen d2b8ab8fe8 Add LLM chat integration with streaming responses via Ollama
Phase 4: Full chat system with SSE streaming, note-aware context, and
conversation persistence.

Backend:
- Migration 0005: conversations + messages tables with FKs and indexes
- Conversation/Message SQLAlchemy models with relationships
- LLM service: ensure_model (auto-pull on startup), stream_chat (NDJSON),
  generate_completion, fetch_url_content (HTML stripping), build_context
  (keyword extraction, related note search, URL content injection)
- Chat service: conversation CRUD, save_response_as_note,
  summarize_conversation_as_note
- Chat routes blueprint: 9 endpoints including SSE streaming for messages,
  save/summarize as note, Ollama model listing
- Auto-pull llama3.1 model on app startup (non-blocking)

Frontend:
- apiStreamPost: SSE client using fetch + ReadableStream
- Chat Pinia store with streaming state management
- ChatView: dedicated /chat page with conversation sidebar + message thread
- ChatPanel: slide-out panel with contextNoteId from current route
- ChatMessage: markdown-rendered message bubble with "Save as Note" action
- Updated AppHeader with Chat nav link + panel toggle button
- Updated App.vue to mount ChatPanel with route-derived context
- Added /chat and /chat/:id routes

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-10 18:45:22 -05:00
bvandeusen 22a3a3c1d1 Initial commit: note-taking/task-tracking app with LLM integration scaffold
Vue 3 + TypeScript frontend with Pinia stores, markdown rendering (marked + DOMPurify),
wikilink/tag linkification, and autocomplete. Quart async backend with SQLAlchemy 2.0,
PostgreSQL ARRAY columns, task-note companion linking, backlinks, and note-to-task
conversion. Docker Compose setup with PostgreSQL 16 and Ollama.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-09 23:35:44 -05:00