Commit Graph

85 Commits

Author SHA1 Message Date
bvandeusen 2cb4e6d3b2 Add version footer, task history UI, and note version retention policy
- App.vue: quiet version footer (fetches /api/version, 0.45 opacity)
- TaskEditorView: History button + HistoryPanel integration (mirrors NoteEditorView)
- note_versions.py: raise MAX_VERSIONS 20→50; add 5-min minimum interval gate
  to prevent autosave exhausting history slots

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-11 16:23:56 -04:00
bvandeusen c7ef709633 Multi-user sharing, groups, and in-app notifications
Backend:
- Alembic migration 0025: groups, group_memberships, project_shares,
  note_shares, notifications tables
- models: Group, GroupMembership, ProjectShare, NoteShare, Notification
- services/access.py: permission resolution (viewer/editor/admin/owner)
- services/groups.py + routes/groups.py: full group CRUD + membership
- services/sharing.py + routes/shares.py: project/note sharing API
- services/notifications.py: in-app notification create/list/mark-read
- routes/in_app_notifications.py: GET/POST notification endpoints
- routes/users.py: user search endpoint
- services/projects.py + services/notes.py: *_for_user variants
- routes updated to use *_for_user on get/list; adds permission field
- app.py: register all new blueprints

Frontend:
- api/client.ts: ShareEntry, GroupEntry, NotificationEntry types + helpers
- stores/notifications.ts: Pinia store with polling
- NotificationBell.vue: bell icon + badge + dropdown toggle + 60s poll
- NotificationsPanel.vue: unread notification list with mark-all-read
- ShareDialog.vue: teleport modal for sharing projects/notes with users/groups
- SharedWithMeView.vue: /shared route listing shared projects and notes
- AppHeader: NotificationBell, Shared nav link
- ProjectView, NoteViewerView, TaskViewerView: Share button + ShareDialog
- SettingsView: Groups admin tab with create/delete groups + member mgmt
- router: /shared route

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-11 15:37:00 -04:00
bvandeusen 878b149f4d feat(backup): v2 backup/restore with all models and FK re-mapping
Export now includes: projects, milestones, task_logs, note_drafts,
note_versions, push_subscriptions; full user fields (oauth_sub,
session_version); full message fields (status, tool_calls).

Restore v2 builds ID maps for all tables and patches FKs in dependency
order (users → projects → milestones → notes → child records). Notes
restored in two passes to handle self-referential parent_id correctly.
v1 backups still restore via _restore_v1 path (parent_id bug also fixed).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-11 11:57:21 -04:00
bvandeusen 810f63e749 Associate research_topic notes with workspace project
run_research_pipeline now accepts project_id; generation_task.py passes
workspace_project_id when the tool is called from a workspace context.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-11 09:29:56 -04:00
bvandeusen 1c6d009603 API: no_project filter + project_id on tasks route
- list_notes(): add no_project param → Note.project_id.is_(None) filter
- GET /api/notes: expose ?no_project=true
- GET /api/tasks: add ?project_id= and ?no_project=true pass-through

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-10 20:04:03 -04:00
bvandeusen f5bee4573f Scope duplicate detection to same type (notes vs tasks)
create_note now only checks against existing notes (is_task=False);
create_task only checks against existing tasks (is_task=True). All three
checks (exact title, fuzzy title, semantic similarity) pass the type
filter through to list_notes and semantic_search_notes.

Adds is_task param to semantic_search_notes() so callers can restrict
results to notes or tasks independently.

Prevents a note titled "Design enemy AI" from blocking a task with the
same title, and vice versa.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-10 08:35:54 -04:00
bvandeusen a90342c9cb Restore semantic duplicate check at 0.90 threshold, 200-char minimum
Raises body character limit from 80 → 200 (skips short/stub notes)
and threshold from 0.87 → 0.90 (only flags near-identical content).

Enemy design notes sharing a template will pass through cleanly;
accidentally duplicated notes with the same detailed content will
still trigger the confirmation prompt.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-10 08:20:20 -04:00
bvandeusen 4dd3c1fe81 Fix queued bubbles, queue persistence, duplicate confirm UX, semantic check
- Queued message bubbles now match user bubble style (primary colour,
  right-aligned, opacity 0.45) in both ChatView and WorkspaceView
- Queue persisted to localStorage (fa_conv_queue_<id>); survives page
  refresh, restored on fetchConversation, cleared on delete
- ToolCallCard confirm/deny: buttons now inline in header row; "Create
  anyway" calls POST /api/notes or /api/tasks directly (no LLM round-trip);
  shows "✓ Created" / "Skipped" state; removed chatStore dependency
- POST /api/notes and POST /api/tasks now accept project (name string) and
  resolve to project_id server-side, matching tools.py behaviour
- Remove semantic similarity duplicate check from create_note and
  create_task — 0.87 threshold was too aggressive for topically-related
  notes; title-based exact/fuzzy checks are sufficient

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-10 08:11:39 -04:00
bvandeusen d7e1fe6aab UX: queue bubbles, dashboard queuing, duplicate confirm buttons, short-note fix
- Dashboard chat: allow typing/sending during streaming (queued in store)
  placeholder updates to "Type to queue…" during generation
- ChatView/WorkspaceView: replace queued chip with actual pending message
  bubbles — light grey, dashed border, "Queued" badge above content; clear
  button below the stack
- ToolCallCard: when tool returns requires_confirmation=True, show
  "Similar content found" label (not "Error"), link to the similar note,
  and "Create Anyway" / "Don't Create" buttons that auto-send the reply
- tools.py: fuzzy title match now returns requires_confirmation=True with
  similar_note data instead of a hard error, so numbered-series notes
  (Lore: X 0, Lore: X 1) can be created with one button click; semantic
  match responses also include similar_note for the link

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-09 23:32:21 -04:00
bvandeusen 690270519f Fix push notifications: focus suppression, empty body, error visibility
- sw.js: suppress notification when the target chat tab is already focused
  (clients.matchAll visibility check before showNotification)
- generation_task.py: provide meaningful body for tool-only responses
  (lists tool names instead of sending an empty string that browsers discard);
  promote scheduling failure from debug to warning
- push.py: promote send errors from warning to error with exc_info;
  log successful sends at INFO so they're visible in normal operation

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-09 22:12:20 -04:00
bvandeusen 3d861d08eb Skip semantic duplicate check for bare-title tasks/notes
The semantic similarity check was flagging unrelated short-title tasks
as duplicates (e.g. "Lore: Shell 0" matching "Lore: Reinitialization 0"
at 91%) because with no body, the embedding is purely title-based and
co-domain tasks in the same project share a tight embedding neighborhood.

Only run the semantic check when the body is ≥ 80 chars — enough
content to make a meaningful comparison. The fuzzy title check already
covers exact/near-exact title duplicates.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-09 18:49:25 -04:00
bvandeusen ef141f07f8 UI polish pass: word count, slash commands, task lists, graph peek, bulk delete, export
- WordCount component (toggle words/chars vs read time, persisted mode)
- TipTap: TaskList/TaskItem extensions, slash command menu (H1-H3, lists, code, quote, task)
- Markdown serializer: task list → `- [ ]` / `- [x]` roundtrip
- GraphView: slide-in peek panel for note/task nodes (body, tags, linked nodes); tag nodes still navigate
- ChatView: bulk-select conversations with two-click confirm delete + chat retention policy (default 90d)
- NoteEditorView: pill tab bar with animated edit/preview toggle + WordCount in toolbar
- WorkspaceNoteEditor: inline search with tag matching, inline new-note creation, WordCount
- WorkspaceTaskPanel: task body rendered in slide-over + Edit link
- Settings: data export (Markdown ZIP / JSON) via GET /api/export
- Accessibility: skip-to-content link, aria-labels on all icon-only buttons
- Fix: export route used non-existent fabledassistant.database — corrected to async_session()
- Fix: ToolCallRecord.status type now includes "running" (was causing TS build error)
- Dockerfile: upgrade npm to latest before install to suppress major-version notice
- npm audit fix: patched minimatch (ReDoS) and rollup (path traversal) — 0 vulnerabilities

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-07 23:15:01 -05:00
bvandeusen 320e879788 Workspace polish: milestone selector, streaming fix, push fix, log quieting
- WorkspaceTaskPanel: add milestone <select> in task detail (PATCH /api/notes/:id),
  replace delete ✕ with trash can SVG icon
- WorkspaceView: scroll to bottom when streaming ends so final message is visible
  without a page refresh
- ToolCallCard: fix search_notes result count (was reading data.total; tool returns
  data.count), so results no longer show "0 found"
- push.py: switch from deprecated WebPusher().send(vapid_private_key=...) to
  webpush() function (pywebpush 2.x API compatibility)
- app.py: downgrade /api/health, /api/chat/status, and static asset requests from
  INFO to DEBUG in after_request logger to reduce log noise at default log level

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-07 12:58:09 -05:00
bvandeusen 74ebb8a87f Project Workspace view, abort button, session invalidation, workspace fixes
Workspace (/workspace/:projectId):
- Three-panel layout (tasks / chat / notes) with CSS grid collapse toggles
- WorkspaceTaskPanel: tasks grouped by milestone, collapsible groups, task
  detail slide-over with status cycling, TaskLogSection work log, and
  inline-confirm delete
- WorkspaceNoteEditor: list view (sorted by updated_at, inline tag pills,
  inline-confirm delete) with editor view (TipTap, TagInput, Suggest tags,
  60s autosave)
- Persistent workspace conversation stored in localStorage per project;
  reused on return visits
- Thinking enabled (think: true) with Reasoning block in streaming bubble
- workspace_project_id backend pipeline: chat.py → generation_task.py →
  llm.py; system prompt uses project title so agent passes project="Title"
  to tools (fixes create_note failing with numeric project string)
- SSE tool-call watcher bridges agent actions to panel updates
- Height fix: workspace-root uses height 100%; app-content switches to
  overflow hidden via :has() selector
- Entry point: "Open Workspace" button on ProjectView

Abort button:
- Stop button in ChatView header and WorkspaceView input bar
- Calls existing cancelGeneration() / POST .../generation/cancel

Session invalidation:
- POST /api/auth/invalidate-sessions bumps session_version, keeps current
  session alive; useful after SSO/OAuth password rotation
- Button in Settings → Active Sessions section

Other:
- Dashboard recent notes limit increased from 8 to 16
- Workspace chat abort replaces Send button while streaming

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-07 11:34:06 -05:00
bvandeusen a8bb687349 Force-directed graph view with tag nodes, project clustering, physics tuning
- New /graph route with D3 force simulation (GraphView.vue)
- Tag nodes as first-class graph nodes (string IDs "tag:name") — clicking
  navigates to /notes?tags=name; tags shown by default
- Invisible project hub nodes attract project members into clusters
- Physics panel with live sliders: repulsion, link distance, link strength,
  project pull, gravity (forceX/forceY, not forceCenter)
- Wikilink edges retain directed arrowheads; tag edges are thin, no arrowhead
- Graph nav link in AppHeader; `g` shortcut in App.vue
- Backend: build_note_graph() emits tag nodes + note→tag edges instead of
  O(n²) note→note shared-tag mesh

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-06 20:50:43 -05:00
bvandeusen b33aa25fb7 Session invalidation on credential change
Add session_version to users table. Sessions now carry session_version
alongside user_id; @login_required rejects any session where the version
doesn't match the DB value.

- migration 0024: session_version INTEGER NOT NULL DEFAULT 1
- models/user.py: session_version Mapped[int] column
- auth.py: version mismatch → session.clear() + 401
- services/auth.py: change_password and reset_password_with_token both
  increment session_version, evicting all other live sessions
- routes/auth.py: login/register/oauth_callback store session_version;
  update_password rehydrates current session with new version so the
  user who changed their password stays logged in

Existing sessions will require one re-login after the migration runs.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-06 16:30:14 -05:00
bvandeusen 48f070f773 Project-aware assist, link suggestions, project-scoped RAG, semantic search tool, SSE race fix
- Writing assistant: inject project notes as context (definition-tagged first), wikilink suggestions
- Link suggestions: server-side endpoint finds unlinked term occurrences, NoteEditorView sidebar panel
- Project-scoped RAG: ChatView ProjectSelector filters semantic+keyword search to selected project
- Semantic search tool: LLM search_notes upgraded to hybrid semantic (0.40 threshold) + keyword merge
- SSE race condition fix: drain remaining events after stream loop exits in chat.py and notes.py
- RAG_AUTO_SNIPPET raised 800→4000; sidebar include uses full note body; MAX_BODY_CHARS 8000→24000
- Enter-to-submit on writing assistant instruction textareas (note and task editors)
- DiffView: equal-line collapsing with 3-line context around changes

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-06 14:02:54 -05:00
bvandeusen 9036dfd931 Note editor sidebar, full-doc assist, persistent drafts, version history
NoteEditorView: two-column sidebar layout (project/milestone/tags/assist
always visible), removed assist toggle button, InlineAssistPanel removed.

Writing assist: whole_doc mode rewrites entire document; DiffView.vue
replaces editor during review showing full-document diff. Scope dropdown
in sidebar switches between whole-document and section modes.

Persistent drafts: migration 0022 adds note_drafts (UNIQUE per note+user)
and note_versions (max 20, auto-pruned) tables. Draft saved after generation
completes, restored on editor mount, cleared on accept/reject. Version
snapshot created automatically whenever note body changes on save.

HistoryPanel.vue: version list + DiffView modal, restore button writes
body back to editor.

Config: OLLAMA_NUM_CTX default raised to 65536; assist num_predict now
tracks Config.OLLAMA_NUM_CTX instead of a hardcoded 4096.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-05 17:10:55 -05:00
bvandeusen b11a92f32d Fix writing assist: disable thinking mode, drop stuck-buffer 409
- stream_chat: add think=False parameter passed through to Ollama payload.
  qwen3 models have thinking enabled by default; without this flag the model
  spends minutes generating internal thinking tokens that stream_chat silently
  discards, leaving the frontend spinner blank until the SSE connection times
  out and the widget disappears.

- create_assist_buffer: orphan (overwrite) a still-running buffer instead of
  raising. The old asyncio task holds a direct reference and completes
  harmlessly against the stale buffer. New requests always win.

- assist_route: remove the 409 guard that blocked new requests when a previous
  generation got stuck. create_assist_buffer now handles this transparently.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-05 13:28:59 -05:00
bvandeusen 9bf047ec45 Task work log, inline writing assistant, task editor sidebar layout
Backend:
- Migration 0021: task_logs table (FK → notes + users, CASCADE, indexed)
- models/task_log.py: SQLAlchemy model with to_dict()
- services/task_logs.py: CRUD with ownership checks, _UNSET sentinel for optional duration clear
- routes/task_logs.py: GET/POST/PATCH/DELETE /api/tasks/<id>/logs
- services/tools.py: log_work LLM tool (resolves task by title, creates log entry)
- services/generation_task.py: retry assist generation up to 3× on HTTP 500

Frontend:
- types/task.ts: TaskLog interface
- TaskLogSection.vue: chronological work log with date+time timestamps, duration badge, inline edit, autofocus
- InlineAssistPanel.vue: streaming preview + diff review rendered inline in editor column
- useAssist.ts: removed chatStore.chatReady gate; toast notifications for errors
- NoteEditorView.vue + TaskEditorView.vue: inline assist panel, aside restricted to idle state
- TaskEditorView.vue: two-column layout (editor+log left, metadata sidebar right), body defaults to Preview, sidebarOpen accordion for mobile
- editor-shared.css: .assist-active-hint style

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-05 13:05:26 -05:00
Bryan Van Deusen eed6aedc9e Add duplicate entry prevention to LLM tool pipeline
Three-layer guard on create_note and create_task:
1. Exact title match (get_note_by_title, case-insensitive) → hard block,
   redirects model to use update_note instead
2. Fuzzy title match (SequenceMatcher ≥82%, punct-stripped word search for
   candidates) → hard block; catches "Game Premise" vs "Game Premise Notes"
3. Semantic content similarity (semantic_search_notes threshold=0.87) → soft
   block with requires_confirmation:true; model asks user then retries with
   confirmed:true; graceful no-op if embedding model is unavailable

create_project and create_milestone now always require confirmed:true before
creation (structural entities — must be intentional). Both are also guarded
by exact + fuzzy title checks post-confirmation.

create_note and create_task gain an optional confirmed parameter (not
required by default; only used to bypass a content-similarity soft-block
after the user has been asked).

Helpers added to tools.py:
- _fuzzy_title_match(title, candidates, threshold=0.82)
- _PUNCT_RE for stripping punctuation before word-search candidate fetch
- difflib.SequenceMatcher and re imported at module level

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-03 14:19:58 -05:00
Bryan Van Deusen 87d3f3ea16 Fix project/milestone association and redesign AppHeader navigation
Backend:
- routes/tasks.py: POST + PUT were silently dropping project_id,
  milestone_id, parent_id from request body — root cause of association
  not saving from the task editor
- routes/tasks.py: GET /api/tasks/:id now includes parent_title in
  response (secondary lookup when parent_id is set)
- routes/notes.py: add PATCH /api/notes/:id for partial updates (used
  by sub-task status toggle; PUT already existed but PATCH was missing)
- routes/projects.py: GET /api/projects/:id/notes now fetches milestone
  IDs and passes them via milestone_ids so tasks assigned to a milestone
  (but lacking project_id) are included in the project view
- services/notes.py: create_note() auto-sets project_id from milestone
  when milestone_id is provided and project_id is omitted; list_notes()
  gains milestone_ids param — when combined with project_id uses OR
  condition (project_id=X OR milestone_id IN (...))

Frontend:
- NoteEditorView: add MilestoneSelector; milestone resets when project
  changes; all save paths (save/create/auto-save) include milestone_id
- stores/notes.ts: add milestone_id to createNote + updateNote types
- TaskEditorView: sub-tasks now inherit milestone_id from parent task
- AppHeader: three-zone layout — brand left, Notes/Projects/Tasks/Chat
  centered (absolute positioning), right rail with status/theme/? and
  gear dropdown containing Settings/Users/Logs; mobile dropdown unchanged

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-03 13:34:01 -05:00
bvandeusen 47c9cb39a2 Auto-generate VAPID keys at startup; no manual configuration needed
On first boot, ensure_vapid_keys() generates a fresh EC P-256 key pair
and saves it to /data/vapid_keys.json (inside the app_data volume) so
it survives container restarts. Subsequent boots load from that file.

VAPID_PRIVATE_KEY / VAPID_PUBLIC_KEY env vars still take precedence for
deployments that prefer to manage keys externally.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-02 23:07:30 -05:00
bvandeusen 4fd2c915b6 Improve project/task/sub-task workflows across backend and web
Backend:
- notes.py: add parent_id filter to list_notes()
- routes/notes.py: expose project_id, milestone_id, parent_id, type
  query params on GET /api/notes
- milestones.py: add find_milestone_by_title() (cross-project search)
- tools.py: list_notes project filter + updated_at; list_tasks milestone
  without project; tag_mode default add; fail-fast project resolution

Web frontend:
- TaskEditorView: add sub-tasks section (fetch, toggle, create inline);
  pre-fill projectId/milestoneId/parentId from URL query params on new task
- ProjectView: add + button in Todo kanban column header linking to
  /tasks/new?projectId=X&milestoneId=Y for quick task creation

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-02 22:00:44 -05:00
bvandeusen 6580d16942 Improve tools reliability and add milestone management to ProjectView
tools.py:
- Default tag_mode changed from 'replace' to 'add' — existing tags are
  preserved unless the user explicitly requests replacement
- create_task, create_note, update_note, create_milestone: project and
  milestone lookup now uses get_by_title with fuzzy fallback; returns a
  clear error (with a hint to use list_projects/list_milestones) instead
  of silently creating duplicate projects or milestones via get_or_create
- create_task: project/milestone resolved before note creation so a bad
  project name fails fast without leaving an orphaned task behind
- update_note return value now includes item_type, tags, project_id, and
  an 'updated' summary string so the LLM can confirm what was modified
- research_topic stub now logs an error and returns failure instead of
  silently returning success when reached via wrong code path

ProjectView.vue:
- Milestone headers now show edit (✎) and delete (✕) action buttons on hover
- Edit triggers inline rename input; Enter/blur commits, Escape cancels
- Delete opens a confirmation modal clarifying tasks are unlinked not deleted

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-02 21:33:13 -05:00
bvandeusen 012eb1d46b Add Projects, Milestones, RAG auto-inject, push notifications, PWA, tag normalisation
## Projects & Milestones (Phases A + G)
- New models: Project, Milestone (Project → Milestone → Task hierarchy)
- notes table: project_id + milestone_id FKs; parent_id FK constraint activated
- Migrations: 0017 (projects), 0018 (push_subscriptions), 0019 (events), 0020 (milestones)
- Services: projects.py, milestones.py (CRUD + progress tracking)
- Routes: /api/projects + /api/projects/<id>/milestones
- LLM tools: create/list/get/update project; create/list milestone; project + milestone + parent_task params on note/task tools
- Frontend: ProjectListView (stacked milestone bars), ProjectView (milestone-grouped kanban), ProjectSelector, MilestoneSelector, NoteEditorView + TaskEditorView updated

## RAG Auto-injection (Phase B)
- Notes ≥0.60 cosine similarity auto-injected into system prompt (max 3, 800 chars each)
- excluded_note_ids param; ChatView "Auto-included" sidebar section

## Summarisation improvements (Phase C)
- Threshold 20→30, keep-recent 6→8, max_tokens 200→400
- Two-pass summarisation for histories >50 messages

## Browser push notifications (Phase E)
- PushSubscription model + migration; pywebpush dependency
- /api/push routes; VAPID config; fire-and-forget on generation complete
- Frontend: sw.js, push store, Settings toggle

## PWA manifest (Phase F)
- manifest.json, Apple meta tags, service worker registration in main.ts

## Tag normalisation
- All tags lowercased + deduplicated at backend (create_note/update_note) and frontend (TagInput sanitize)
- Note/Task types gain project_id + milestone_id fields; store signatures updated

## CalDAV
- Radicale embedded server reverted; back to user-configured external CalDAV

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-02 20:52:21 -05:00
bvandeusen 3d7be5888e Remove intent model entirely; quick-capture uses primary model
The separate intent model (OLLAMA_INTENT_MODEL / qwen2.5:7b) is removed
from every part of the system. All classification now uses the primary model.

Changes:
- config.py: remove OLLAMA_INTENT_MODEL
- intent.py: remove classify_intent() and all supporting infrastructure
  (_SYSTEM_PROMPT_TEMPLATE, _RESEARCH_PREFIX, _PRIOR_WORK_REFS); file now
  only contains the quick-capture classifier
- quick_capture.py: classify_capture_intent() now called with Config.OLLAMA_MODEL
- generation_task.py: remove intent_model_setting DB lookup and get_setting import;
  history summarization and research pipeline use the primary model directly
- research.py: remove intent_model parameter from run_research_pipeline() and
  _generate_sub_queries(); both use the model param throughout
- routes/settings.py: remove intent_model from model-key validation and response
- app.py: remove intent model pre-warming at startup
- SettingsView.vue: remove Intent Model selector and related refs/state

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-02 18:41:49 -05:00
bvandeusen 53e54ea761 Remove intent router from chat pipeline; raise OLLAMA_NUM_CTX to 16384
The intent classifier (Phase 21) is removed from the main chat generation
path. The main model now handles all tool routing natively via Ollama's
structured tool-calling API, eliminating misidentification issues caused
by the small intent model.

Changes:
- generation_task.py: remove classify_intent call, intent_task, _WRITE_TOOLS,
  _TOOL_ACTIONS, _INTENT_TRIGGER_WORDS, _should_skip_intent(), and the entire
  round-0 intent-first + write-tool confirmation block (~315 lines removed)
- research_topic tool calls are now handled inline in the streaming loop:
  runs run_research_pipeline, streams synthesis to buf, then breaks the round
  loop (research is still the full response, no model follow-up)
- config.py: raise OLLAMA_NUM_CTX default from 8192 to 16384

The quick-capture dedicated classifier (classify_capture_intent) is unchanged.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-02 18:30:21 -05:00
bvandeusen 2c2874a1cc Fix streaming timeout and false-offline status indicator
- stream_chat and stream_chat_with_tools: remove read=300s per-chunk
  timeout, replace with read=None. In httpx streaming mode, the read
  timeout applies per-chunk — if Ollama pauses >300s while processing
  a large input context before the first token, it raises ReadTimeout,
  killing generation and leaving the assistant message as an empty stub.
  With read=None the stream is unbounded; connect=30s still guards the
  initial connection.

- chat_status_route: increase Ollama status check timeout 5s → 10s.
  When Ollama is busy processing a large prompt it can be slow to
  respond to /api/tags, causing the status indicator to briefly flip to
  "offline" even though generation is running normally.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-02 17:50:42 -05:00
bvandeusen 6f5170854b Remove CalDAV todo tools; overhaul quick-capture
- Remove all 6 CalDAV todo tools (create/list/update/complete/delete/search_todos)
  from tools.py definitions, imports, execute_tool branches, intent routing rules,
  generation_task labels/actions, and llm.py system prompt hints. CalDAV event
  tools remain. Todo functions still exist in caldav.py but are no longer exposed.

- Quick-capture now uses a dedicated classify_capture_intent() with a focused
  _CAPTURE_SYSTEM_PROMPT that always routes to a tool (never null). Tool set
  expanded: create_note/task/event + update_note + research_topic.

- research_topic in quick-capture calls run_research_pipeline() directly (no SSE
  buffer). run_research_pipeline() now accepts buf=None; all buf.append_event
  calls are guarded so status events are skipped when no buffer is provided.

- Fallback note now always sets body=text (was empty for texts ≤80 chars).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-01 19:13:07 -05:00
bvandeusen 1890f0c7f1 Fix image search: embed markdown instead of describing URLs
- tools.py: search_images result now includes 'embed' (ready-to-use
  markdown image syntax) and 'citation' fields instead of raw 'local_url';
  adds 'instructions' field so the model knows to render them verbatim
- llm.py: system prompt now explicitly tells the model to embed images
  using the 'embed' field rather than describing or listing URLs
- markdown.ts: explicitly allow src/alt in PURIFY_OPTS_FULL so img tags
  are never stripped by DOMPurify

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-01 13:19:59 -05:00
bvandeusen efe0a15b8c Add image search with local cache (Phase 23)
Images found via SearXNG are fetched server-side, stored on disk, and
served from /api/images/<id> — the user's browser never contacts the
original image host.  Original URLs are preserved for citation.

New files:
- alembic/versions/0016_add_image_cache.py  — image_cache table
- src/fabledassistant/models/image_cache.py — SQLAlchemy model
- src/fabledassistant/services/images.py    — fetch/store/serve logic
- src/fabledassistant/routes/images.py      — GET /api/images/<id>

Modified:
- config.py: IMAGE_CACHE_DIR (/data/images), IMAGE_MAX_BYTES (5 MB)
- research.py: _search_searxng_images() — SearXNG categories=images
- tools.py: _IMAGE_TOOLS def + search_images branch in execute_tool
- intent.py: search_images routing rule (explicit visual language only)
- app.py: register images_bp
- docker-compose.yml: image_cache named volume mounted at /data/images
- ToolCallCard.vue: "image_search" label

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-01 12:55:10 -05:00
bvandeusen 90afd3f131 Improve suggested notes: limit 8, threshold 0.45, show relevance scores
- Raise similarity threshold 0.30 → 0.45: only genuinely relevant notes
  shown; loosely-related notes no longer pad the sidebar
- Increase max suggested notes 3 → 8 (zero added compute — threshold is
  the real gate; the embedding call is fixed regardless of limit)
- semantic_search_notes now returns list[tuple[float, Note]] instead of
  list[Note] so scores propagate through context_meta to the frontend
- Keyword fallback notes carry score=null (no cosine similarity available)
- ChatView sidebar shows % badge on each suggested note:
  green ≥75%, amber 60–74%, muted <60%
  Hovering reveals the raw score in a tooltip

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-01 12:10:39 -05:00
bvandeusen 5c043b76b6 Validate create_note/create_task title is a string before DB insert
The model occasionally passes title as a list (e.g. when asked to create
multiple notes at once), causing asyncpg DataError on the INSERT. Return a
clear error result so the model sees the problem and retries with individual
calls instead of crashing with an unhandled exception.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-28 20:01:43 -05:00
bvandeusen 4998d04739 Fix search_web over-triggering when user references existing notes
Add a Python fast-path regex (_PRIOR_WORK_REFS) in classify_intent that
detects phrases like "research you did", "note you made", "using your
research", "based on the research" etc. and returns no-tool immediately —
saving the 19s intent LLM call and correctly letting the main model answer
using search_notes/context rather than firing off a web search.

Also tighten the intent prompt rules for search_web: explicitly prohibit
using it for creative/brainstorming requests or when the user references
existing notes, and add a rule that creative/ideation questions ("think of",
"come up with", "brainstorm") always route to null (chat).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-28 18:12:58 -05:00
bvandeusen fb3a6d2445 Fix 'default' model selection breaking readiness indicator
When a user selected the 'Default' option in Settings, the dropdown
sent an empty string "" to the backend. The route saved it as a DB row,
which caused get_setting() to return "" instead of falling back to
Config defaults. The chat status endpoint then tried to match "" against
installed model names — always failing — resulting in model: "not_found"
and a permanently failing readiness indicator.

services/settings.py:
- Add delete_setting() helper: removes a setting row so get_setting()
  correctly falls back to its hardcoded default argument

routes/settings.py:
- Import delete_setting
- When default_model or intent_model are saved as empty string, delete
  the DB row instead of storing "" — cleanly restores Config fallback

routes/chat.py:
- chat_status_route: add explicit `or Config.OLLAMA_MODEL` guard for
  any existing "" rows written before this fix (migration safety net)
- send_message and summarize routes: same guard on model resolution
  so empty settings never cause silent generation failures

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-28 16:06:02 -05:00
bvandeusen 926e4bb570 Upgrade intent model to qwen2.5:7b, simplify intent prompt rules
config.py:
- Default OLLAMA_INTENT_MODEL: qwen2.5:1.5b → qwen2.5:7b
- Startup will auto-pull and warm the new model on next container restart

intent.py:
- Replaced phrase-matching examples in search_web and research_topic rules
  with semantic descriptions. The 7B model doesn't need example phrases to
  understand intent — it can reason from the tool's purpose. Removes implied
  usage patterns that caused misclassifications on conversational phrasing
  (e.g. "I've been thinking about buying shirts, can you research this?").
- research_topic rule now explicitly covers any subject regardless of phrasing,
  including shopping decisions, comparisons, how-things-work questions, etc.
- search_web rule clarified as "short summary, no note" vs research_topic's
  "comprehensive written reference"

The 1.5B model required prescriptive phrase examples to route correctly; the
7B model has sufficient language understanding to classify from semantic intent.
Expected improvement: ~1-2s intent calls (vs 0.4-9s for the 1.5B model which
sometimes timed out or misclassified longer/conversational messages).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-28 15:51:27 -05:00
bvandeusen d1c7ce88df Fix intent classifier missing Research button requests
Two fixes for the intent model failing to route 'Research: X' messages
to research_topic:

1. Fast-path in classify_intent: if the message matches ^Research:\s+.+
   (the exact format the UI Research button always sends), skip the LLM
   call entirely and return research_topic with high confidence. This is
   100% reliable and saves an unnecessary model call for this pattern.

2. Expanded research_topic rule examples in the system prompt to include
   "Research: X" prefix format, shopping-style queries ("research where
   to buy X"), and clarification that the topic is everything after the
   keyword — improves LLM routing for natural-language research requests
   that don't match the previous narrow examples.

Root cause: qwen2.5:1.5b misclassified "Research: where to buy three-
quarter sleeve tee shirts" as general chat (shopping query phrasing
combined with the colon confused the small model).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-28 13:07:39 -05:00
bvandeusen a95d17fc04 Fix research_topic loop when intent misses and main model calls tool directly
When the intent model doesn't classify a research request (low confidence,
long message, etc.), the main model (qwen3) would correctly identify
research_topic itself and call it via the streaming tool loop. But
execute_tool("research_topic") only returns a dummy research_pending
placeholder, causing the model to see the result and retry — looping
up to MAX_TOOL_ROUNDS times.

Fix: filter research_topic out of stream_tools (the tool list given to
the main model via stream_chat_with_tools). research_topic is an
intent-only routing tool; the main model should never call it directly.
The full tools list (including research_topic) is still passed to
classify_intent so intent routing continues to work.

The _INTENT_ONLY_TOOLS frozenset makes this pattern explicit and
extensible for future intent-only tools.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-28 13:05:36 -05:00
bvandeusen df4c52412d Phase 22b: Parallel research fetching, streaming synthesis, intent optimizations
research.py:
- Parallelize all 5 SearXNG queries concurrently (200ms stagger via asyncio.gather)
- Parallelize all URL fetches in parallel (asyncio.gather) — up to 15 URLs at once
  instead of sequential fetches; biggest performance win (was O(n) × 15s, now ~15s flat)
- _synthesize_note accepts buf: when provided uses stream_chat (num_ctx=16384,
  num_predict=8192) to emit tokens into the chat buffer in real time so users see
  the note being written; falls back to generate_completion when buf=None
- Added \n\n---\n\n separator before "Research complete!" to cleanly mark boundary
  after streamed synthesis content

intent.py:
- classify_intent passes num_ctx=4096 to generate_completion — reduces VRAM pressure
  and prefill time for the intent model call on every single request

generation_task.py:
- _INTENT_TRIGGER_WORDS frozenset (~50 action/object/date words) + _should_skip_intent()
  skips intent classification for short messages (≤10 words) with no trigger words;
  saves 400-800ms model call for conversational replies ("thanks", "okay", etc.)
- Added \n\n---\n\n separator before research "done" text in research_topic branch

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-27 18:24:15 -05:00
bvandeusen 590682a5d2 Phase 22: SearXNG web research pipeline + settings layout overhaul
Research pipeline (research_topic tool):
- New service: services/research.py — sub-query generation, SearXNG
  search, URL fetch, deduplication, and LLM synthesis into a note
- 5 sub-queries × 3 pages = up to 15 sources, capped at 12 for synthesis
- Synthesis uses num_ctx=16384 + max_tokens=8192 for long-form output
- Prompt demands 2500+ words, 6+ topic-appropriate sections, detailed prose
- 429 retry with backoff; 1s inter-query sleep; raw_decode JSON parsing

search_web tool (new):
- Lightweight single-query SearXNG search, results returned inline in chat
- LLM answers conversationally in round 1; no note created
- web_search result type with external links in ToolCallCard

Infrastructure:
- llm.py: generate_completion accepts num_ctx override
- config.py: SEARXNG_URL + Config.searxng_enabled()
- docker-compose: OLLAMA_NUM_PARALLEL=2, commented SEARXNG_URL example
- intent.py: search_web and research_topic routing rules

Settings UI:
- 2-column grid layout (small sections pair up, complex span full width)
- Search Test section: live SearXNG query with result preview
- GET /api/settings/search?q= proxy endpoint
- Research button (magnifier) in ChatView input toolbar → popover modal

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-27 15:21:38 -05:00
bvandeusen 432e0bd2a0 Show Qwen3 thinking output in chat as collapsible Reasoning block
Ollama streams message.thinking tokens alongside message.content when
think=True — previously silently dropped. Now forwarded end-to-end.

Backend:
- llm.py: ChatChunk type gains "thinking" variant; stream_chat_with_tools
  yields ChatChunk(type="thinking") for msg.thinking chunks before content
- generation_task.py: thinking chunks emit "thinking_chunk" SSE events
  (not added to content_so_far — not persisted to DB)

Frontend:
- types/chat.ts: Message.thinking?: string (session-only, not from DB)
- stores/chat.ts: streamingThinking ref; thinking_chunk handler accumulates
  chunks; on done, thinking carried into committed Message object then cleared
- ChatMessage.vue: collapsible <details class="thinking-block"> shown for
  messages that have .thinking content (collapsed by default)
- ChatView.vue + ChatPanel.vue: live thinking block in streaming bubble —
  open while only thinking is flowing, auto-collapses when content arrives;
  typing indicator hidden while thinking is active

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-26 23:16:59 -05:00
bvandeusen 5e83c8a56d Add explicit warm-wait before generation starts
Instead of relying solely on retry-on-500, poll /api/ps before starting
any LLM stream so the main model has time to fully load into VRAM.

- llm.py: add wait_for_model_loaded(model, timeout=90s) — polls /api/ps
  every 2s, returns True when model appears in loaded list
- generation_task.py: launch model_load_task in parallel with build_context
  and classify_intent (both use fast/small-model ops that don't need the
  main model); after context is built, await the load task — shows
  "Loading model..." status only if the user actually has to wait;
  logs a warning and proceeds if 90s timeout elapses

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-26 22:49:06 -05:00
bvandeusen e119331645 Phase 21: Intent-first pipeline, visible ack, KV-stable system prompt
Pipeline changes (generation_task.py, intent.py):
- Remove optimistic streaming queue/race (_drain_queue deleted)
- Remove _generate_acknowledgment — ack now embedded in intent JSON
- Round 0: await intent (~400ms), stream ack immediately as TTFT,
  then execute tool sequentially; chat-only streams directly
- IntentResult.ack: one-sentence acknowledgment, intent max_tokens 200→350
- _parse_intent extracts and trims ack field

KV cache stability (llm.py, generation_buffer.py, generation_task.py):
- build_context: replace cached_note_ids with include_note_ids
- Auto-found notes populate context_meta["auto_notes"] for sidebar but
  are NOT injected into system prompt (--- Related Notes --- removed)
- Explicitly included notes injected as --- Included Notes ---
- _conv_note_cache dict + get/set/clear functions removed from generation_buffer.py
- All clear_conv_note_cache() calls removed

Cold model retry (llm.py):
- generate_completion (used by classify_intent) retries on HTTP 500:
  3 attempts with 3s/6s delays — prevents intent failure during cold load

API + frontend (routes/chat.py, stores/chat.ts, views/ChatView.vue, components/ChatPanel.vue):
- exclude_note_ids → include_note_ids throughout
- ChatView sidebar: Suggested (auto-found, + to include) + In Context (× to remove)
- ChatPanel: remove exclude button from context pills; no IDs passed to sendMessage

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-26 22:34:54 -05:00
bvandeusen 316a85e13b Phase 20: Dedicated tag field — chip input, explicit tags array
Tags are now a first-class field rather than being auto-extracted from
the note body. A new TagInput.vue chip component handles tag entry in
both editor views with autocomplete, Enter/comma/backspace UX, and
space-to-hyphen sanitization.

Backend:
- routes/notes.py: create reads tags from JSON; update accepts explicit
  tags (omit = keep existing); append_tag writes to tags array with
  dedup; suggest-tags accepts current_tags filter; remove extract_tags
- routes/tasks.py: same — explicit tags on create/update; remove extract_tags
- services/tag_suggestions.py: current_tags param replaces body extraction
- services/tools.py: create_note tool schema adds tags param; executor passes it
- services/llm.py: system prompt tells LLM to use tags param, not embed #tag in body

Frontend:
- components/TagInput.vue: new chip-based tag input (autocomplete, keyboard UX)
- NoteEditorView.vue / TaskEditorView.vue: tags ref loaded from note.tags;
  TagInput placed between title and body; save/autosave include tags; suggest
  now adds chips; fetchTagSuggestions passes current_tags; dirty tracks tags
- TiptapEditor.vue: remove fetchTags prop and TagSuggestion extension;
  keep TagDecoration for legacy inline #tag highlighting

No DB migration needed — tags column already correct.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-26 06:36:35 -05:00
bvandeusen 7947134e22 Fix tag handling for multi-word tags
Tags with spaces (e.g. #science fiction) were breaking extraction because
TAG_RE only matched word characters — it would stop at the space and extract
#science instead of #science-fiction.

- TAG_RE (backend + frontend): add hyphens to character class so #science-fiction
  is recognized as a single tag: [\w][\w-]* per segment
- System prompt: instruct LLM to use hyphens in multi-word tags, never spaces
- tag_suggestions.py: update prompt example + sanitize output by replacing
  spaces with hyphens as a safety net regardless of LLM output
- append-tag route: sanitize incoming tag (spaces → hyphens) before appending

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-26 06:05:58 -05:00
bvandeusen d5a5373872 Add stream retry to all generation paths, not just round 0
Adds _stream_with_retry() async generator (wraps stream_chat_with_tools
with up to 2 retries on Ollama 500, 3s/6s delay). Previously only the
optimistic round 0 _fill_queue had retry logic. Two paths were still
bare: the declined-write-tool fresh stream, and the round 1+ stream.

Round 1 500s occur when tag suggestions (fire-and-forget inside
execute_tool) race the follow-up stream to the same model. The retry
waits for tag suggestions to complete before succeeding.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-26 05:55:09 -05:00
bvandeusen fc7b2e7305 Retry Ollama 500 errors in optimistic stream with backoff
With optimistic streaming, intent (qwen2.5:1.5b) and the main stream
(qwen3:latest) start concurrently. When both models are cold-loading,
Ollama returns 500 for both simultaneously. The intent 500 was already
handled silently in classify_intent; the stream 500 now retries up to
2 times (3s then 6s delay) before propagating as an error. 500s only
occur on the first cold-load pair — subsequent requests hit warm models.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-25 23:15:05 -05:00
bvandeusen 98d3fca277 Implement optimistic streaming to eliminate intent classification latency
Start the main LLM stream immediately after build_context finishes instead
of waiting for intent classification to complete. Race the two concurrently:

- Intent wins before first token → cancel stream, execute tool (tool path
  unchanged: confirmation, acknowledgment, multi-round loop all preserved)
- First token wins → discard intent, user sees output immediately

For pure chat messages (no tool needed, the common case) this eliminates
the full intent classification RTT from TTFT. For tool calls, intent
typically wins the race since it finishes before the main model produces
its first token, so tool behaviour is unchanged in practice.

Also extracts _drain_queue() as a module-level async generator helper.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-25 23:03:30 -05:00
bvandeusen 7b6248c8a7 Add OLLAMA_NUM_CTX config to reduce VRAM usage
Replaces the hardcoded num_ctx=32768 KV cache allocation with a
configurable env var defaulting to 8192. This significantly reduces
VRAM pressure when multiple services share the GPU.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-25 22:02:06 -05:00