docs: add missing content from summary.md (API reference, Android app, file structure, LLM internals)

- docs/api-reference.md: complete REST API endpoint reference (~60+ routes)
- docs/android-app.md: Flutter companion app stack, architecture, feature status
- docs/architecture.md: detailed file-by-file reference for all backend services and frontend components
- docs/features.md: LLM pipeline internals (intent routing, tool loop, duplicate guards, image search, research pipeline), roadmap
- docs/development.md: full migration chain (0001–0026) with naming and caveats
- README.md: link to new api-reference and android-app docs

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
2026-03-24 21:25:12 -04:00
parent 47e248d9ac
commit e2133529a0
6 changed files with 492 additions and 0 deletions
+73
View File
@@ -0,0 +1,73 @@
# Android Companion App
The Android companion app lives in a separate repository at `/home/bvandeusen/Nextcloud/Projects/fabled_app`.
## Stack
- Flutter + Dart
- Riverpod (state management)
- GoRouter (navigation)
- Dio (HTTP client)
- PersistCookieJar (session persistence)
- SSE streaming via `fetch` + `ReadableStream` bridge
## Architecture
```
lib/
app.dart # GoRouter + _Shell + _QuickCaptureBar
core/constants.dart # Routes.*
data/
models/ # note.dart, task.dart, project.dart
api/ # notes_api.dart, tasks_api.dart, projects_api.dart
repositories/ # notes, tasks, projects repositories
providers/
api_client_provider.dart # all API + repository providers
notes_provider.dart # NotesNotifier
tasks_provider.dart # TasksNotifier
projects_provider.dart # ProjectsNotifier
screens/
notes/note_edit_screen.dart # chip tag input + ProjectSelector
tasks/task_edit_screen.dart # ProjectSelector
projects/project_list_screen.dart
widgets/
project_selector.dart # reusable DropdownButtonFormField
```
## Navigation
4-tab shell (Notes · Tasks · Projects · Chat):
- Phone: bottom `NavigationBar`
- Tablet/landscape: `NavigationRail`
Quick Capture bar persists across all tabs. Settings accessible from top-right icon.
## Feature Status
| Feature | Status | Notes |
|---------|--------|-------|
| Notes CRUD | ✅ | Tags chip input; project selector in editor |
| Tasks CRUD | ✅ | Project selector in editor |
| Projects list | ✅ | Active/archived sections; long-press status change; create dialog |
| Chat + SSE | ✅ | Full streaming |
| Quick Capture | ✅ | Offline queue with retry |
| Tags | ✅ | Chip input in NoteEditScreen; typed as `List<String>` |
| Project assignment | ✅ | `ProjectSelector` dropdown in Note + Task editors |
| Milestones | ❌ deferred | Too granular for mobile; web UI handles it |
| Push notifications | ❌ incompatible | Backend uses browser VAPID; Flutter needs FCM/APNs — separate implementation required |
| CalDAV settings | ❌ intentional | Server-side config only; not exposed in mobile app |
## API Compatibility Notes
- `GET /api/projects/:id` returns a flat JSON object (not `{project: ...}` wrapper); includes `summary` field.
- `POST /api/projects` returns the project dict directly (201).
- `PATCH /api/projects/:id` returns the updated project dict.
- Task body field is `body` (not `description`) — the app maps `description``body` on serialize.
## Self-Update
The app supports self-update via the Forgejo release API (`update_provider.dart`). It checks the latest release tag and prompts the user to download and install a new APK when one is available.
## CI
Builds are triggered from the Forgejo Actions pipeline in the `fabled_app` repository. The APK is attached to the release as a downloadable artifact.
+244
View File
@@ -0,0 +1,244 @@
# API Reference
All endpoints require login (session cookie or `Authorization: Bearer <api-key>`) unless marked **(public)**.
## Health
| Method | Path | Description |
|--------|------|-------------|
| GET | `/api/health` | Health check **(public)** |
## Auth
| Method | Path | Description |
|--------|------|-------------|
| GET | `/api/auth/status` | `{has_users, registration_open, oauth_enabled, local_auth_enabled}` **(public)** |
| POST | `/api/auth/register` | Register new user (first user becomes admin; 403 if registration closed or local auth disabled) |
| POST | `/api/auth/login` | Login with username/password (403 if local auth disabled) |
| POST | `/api/auth/logout` | Clear session |
| GET | `/api/auth/me` | Current user info (includes `has_password: bool`) |
| PUT | `/api/auth/password` | Change password `{current_password, new_password}` |
| PUT | `/api/auth/email` | Change email `{email, password?}` (password required only for local-auth users) |
| POST | `/api/auth/invalidate-sessions` | Bump `session_version` — evicts all other sessions, keeps current alive |
| POST | `/api/auth/forgot-password` | Send password reset email `{email}` |
| POST | `/api/auth/reset-password` | Reset password with token `{token, new_password}` |
| GET | `/api/auth/oauth/login` | Initiate OIDC PKCE flow → redirect to provider |
| GET | `/api/auth/oauth/callback` | OIDC callback — exchange code, find/create user, redirect to `/` |
| GET | `/api/auth/invitation/:token` | Validate invitation token **(public)** |
| POST | `/api/auth/register-with-invite` | Register with token `{token, username, password}` **(public)** |
## Notes
| Method | Path | Description |
|--------|------|-------------|
| GET | `/api/notes` | List notes. Params: `q`, `tag`, `sort`, `order`, `limit`, `offset`, `project_id`, `milestone_id`, `parent_id`, `type` (`note`/`task`/`all`) |
| POST | `/api/notes` | Create note `{title, body, tags?, status?, priority?, due_date?, project_id?, milestone_id?, parent_id?}` |
| GET | `/api/notes/tags` | All tags (param: `q` for filter) |
| POST | `/api/notes/suggest-tags` | LLM tag suggestions `{title, body, current_tags?}``{suggested_tags}` |
| POST | `/api/notes/link-suggestions` | Detect note titles as plain text in body `{body, project_id, exclude_note_id}``[{note_id, title, count}]` |
| GET | `/api/notes/by-title` | Resolve note by exact title (param: `title`) |
| POST | `/api/notes/resolve-title` | Get-or-create note by title `{title}` (wikilink click) |
| GET | `/api/notes/:id` | Get single note |
| PUT | `/api/notes/:id` | Full update |
| PATCH | `/api/notes/:id` | Partial update (same fields as PUT) |
| DELETE | `/api/notes/:id` | Delete note |
| POST | `/api/notes/:id/convert-to-task` | Set `status='todo'`, `priority='none'` |
| POST | `/api/notes/:id/convert-to-note` | Clear `status`, `priority`, `due_date` |
| POST | `/api/notes/:id/append-tag` | Add tag `{tag}` → updated note |
| GET | `/api/notes/:id/backlinks` | Notes/tasks with `[[Title]]` references to this note |
| GET | `/api/notes/:id/versions` | List note version history |
| GET | `/api/notes/:id/versions/:vid` | Get a specific version |
| GET | `/api/notes/:id/draft` | Get current AI draft |
| PUT | `/api/notes/:id/draft` | Save AI draft |
| DELETE | `/api/notes/:id/draft` | Delete AI draft |
| POST | `/api/notes/assist` | Launch AI assist generation → 202 `{body, target_section?, instruction, whole_doc?}` |
| GET | `/api/notes/assist/stream` | SSE stream for assist (Last-Event-ID reconnect; events: `chunk`, `done`, `error`) |
## Tasks
| Method | Path | Description |
|--------|------|-------------|
| GET | `/api/tasks` | List tasks. Params: `q`, `tag`, `status`, `priority`, `due_before`, `due_after`, `sort`, `order`, `limit`, `offset` |
| POST | `/api/tasks` | Create task (accepts `project` name string → resolved to `project_id`) |
| GET | `/api/tasks/:id` | Get task (includes `parent_title`) |
| PUT | `/api/tasks/:id` | Full update |
| PATCH | `/api/tasks/:id/status` | Quick status update `{status}` |
| DELETE | `/api/tasks/:id` | Delete task |
| GET | `/api/tasks/:id/logs` | List work logs |
| POST | `/api/tasks/:id/logs` | Create log `{content, duration_minutes?}` |
| PATCH | `/api/tasks/:id/logs/:log_id` | Update log |
| DELETE | `/api/tasks/:id/logs/:log_id` | Delete log |
## Projects & Milestones
| Method | Path | Description |
|--------|------|-------------|
| GET | `/api/projects` | List projects (owned + shared) |
| POST | `/api/projects` | Create project |
| GET | `/api/projects/:id` | Get project with `milestone_summary` |
| PATCH | `/api/projects/:id` | Update project |
| DELETE | `/api/projects/:id` | Delete project |
| GET | `/api/projects/:id/notes` | Notes + tasks in this project |
| GET | `/api/projects/:id/milestones` | List milestones |
| POST | `/api/projects/:id/milestones` | Create milestone |
| PATCH | `/api/projects/:id/milestones/:mid` | Update milestone |
| DELETE | `/api/projects/:id/milestones/:mid` | Delete milestone |
## Sharing
| Method | Path | Description |
|--------|------|-------------|
| GET | `/api/projects/:id/shares` | List project shares |
| POST | `/api/projects/:id/shares` | Create project share `{user_id?, group_id?, permission}` |
| PATCH | `/api/projects/:id/shares/:sid` | Update permission |
| DELETE | `/api/projects/:id/shares/:sid` | Remove share |
| GET | `/api/notes/:id/shares` | List note shares |
| POST | `/api/notes/:id/shares` | Create note share |
| PATCH | `/api/notes/:id/shares/:sid` | Update permission |
| DELETE | `/api/notes/:id/shares/:sid` | Remove share |
| GET | `/api/shared-with-me` | All resources shared with the current user |
## Groups
| Method | Path | Description |
|--------|------|-------------|
| GET | `/api/groups` | List all groups (admin only) |
| POST | `/api/groups` | Create group `{name, description?}` |
| PATCH | `/api/groups/:id` | Update group |
| DELETE | `/api/groups/:id` | Delete group |
| GET | `/api/groups/:id/members` | List members |
| POST | `/api/groups/:id/members` | Add member `{user_id, role}` |
| PATCH | `/api/groups/:id/members/:uid` | Update member role |
| DELETE | `/api/groups/:id/members/:uid` | Remove member |
## Chat
| Method | Path | Description |
|--------|------|-------------|
| GET | `/api/chat/conversations` | List conversations (params: `limit`, `offset`) |
| POST | `/api/chat/conversations` | Create conversation `{title?, model?}` |
| POST | `/api/chat/conversations/bulk-delete` | Delete multiple conversations `{ids: number[]}` |
| GET | `/api/chat/conversations/:id` | Get conversation with all messages |
| PATCH | `/api/chat/conversations/:id` | Update title or model |
| DELETE | `/api/chat/conversations/:id` | Delete conversation (cascades to messages) |
| POST | `/api/chat/conversations/:id/messages` | Start generation → 202. Body: `{content, context_note_id?, include_note_ids?, rag_project_id?, workspace_project_id?, think?}` |
| GET | `/api/chat/conversations/:id/generation/stream` | SSE stream (Last-Event-ID reconnect; events: `context`, `chunk`, `tool_call`, `status`, `done`, `error`) |
| POST | `/api/chat/conversations/:id/generation/cancel` | Cancel active generation |
| POST | `/api/chat/messages/:id/save-as-note` | Save assistant message as note |
| POST | `/api/chat/conversations/:id/summarize` | Summarize conversation → note |
| GET | `/api/chat/status` | Ollama availability + model state `{ollama, model, default_model}` |
| GET | `/api/chat/models` | List installed Ollama models (includes `loaded: bool`, `modified_at`) |
| POST | `/api/chat/models/pull` | Pull model (SSE NDJSON progress) `{model}` |
| POST | `/api/chat/models/delete` | Delete model `{model}` |
| GET | `/api/chat/ps` | Currently loaded (hot) models |
| POST | `/api/chat/warm` | Pre-load model into VRAM `{model}` → 202 |
## Quick Capture
| Method | Path | Description |
|--------|------|-------------|
| POST | `/api/quick-capture` | Classify + create item from natural language `{text}``{success, type, message, data}` |
## Search
| Method | Path | Description |
|--------|------|-------------|
| GET | `/api/search` | Semantic + keyword search across notes and tasks. Params: `q`, `type` (`note`/`task`/`all`), `limit` |
## Briefing
| Method | Path | Description |
|--------|------|-------------|
| GET | `/api/briefing/config` | Get briefing configuration |
| PUT | `/api/briefing/config` | Save briefing configuration |
| GET | `/api/briefing/feeds` | List RSS feeds |
| POST | `/api/briefing/feeds` | Add RSS feed `{url, name?, category?}` |
| DELETE | `/api/briefing/feeds/:id` | Delete feed |
| POST | `/api/briefing/feeds/refresh` | Trigger immediate feed refresh → `{feeds_refreshed, new_items}` |
| GET | `/api/briefing/weather` | Get weather configuration |
| PUT | `/api/briefing/weather` | Save weather locations |
| POST | `/api/briefing/weather/geocode` | Geocode address `{query}``{lat, lon, label}` |
| POST | `/api/briefing/trigger` | Manually fire a briefing slot `{slot}` |
| GET | `/api/briefing/conversations` | List past briefing conversations |
| GET | `/api/briefing/conversations/today` | Get/create today's briefing conversation |
| GET | `/api/briefing/conversations/:id/messages` | Get messages for a briefing conversation |
## Settings
| Method | Path | Description |
|--------|------|-------------|
| GET | `/api/settings` | All settings as `{key: value}` |
| PUT | `/api/settings` | Update settings `{key: value, ...}` |
| GET | `/api/settings/models` | Installed models + defaults |
| GET | `/api/settings/search` | Proxy SearXNG search (params: `q`) |
## API Keys
| Method | Path | Description |
|--------|------|-------------|
| GET | `/api/api-keys` | List user's API keys |
| POST | `/api/api-keys` | Create key `{name, scope}``{key, ...}` (key shown once) |
| DELETE | `/api/api-keys/:id` | Revoke key |
## Fable MCP Distribution
| Method | Path | Description |
|--------|------|-------------|
| GET | `/api/fable-mcp/info` | `{available: bool, filename: string\|null}` |
| GET | `/api/fable-mcp/download` | Download wheel file |
## Notifications
| Method | Path | Description |
|--------|------|-------------|
| GET | `/api/notifications` | List notifications |
| GET | `/api/notifications/count` | Unread count |
| POST | `/api/notifications/:id/read` | Mark read |
| POST | `/api/notifications/read-all` | Mark all read |
## Push
| Method | Path | Description |
|--------|------|-------------|
| GET | `/api/push/vapid-public-key` | VAPID public key for subscription |
| POST | `/api/push/subscribe` | Register push subscription |
| DELETE | `/api/push/subscribe` | Unregister push subscription |
## Images
| Method | Path | Description |
|--------|------|-------------|
| GET | `/api/images/:id` | Serve cached image **(no auth required — IDs are opaque SHA-256)** |
## Users
| Method | Path | Description |
|--------|------|-------------|
| GET | `/api/users/search` | Search users by username/email prefix (param: `q`, min 2 chars, excludes self) |
## Export
| Method | Path | Description |
|--------|------|-------------|
| GET | `/api/export` | Export data. Params: `format=markdown` (ZIP with `.md` + YAML frontmatter) or `format=json` |
## Admin
| Method | Path | Description |
|--------|------|-------------|
| GET | `/api/admin/backup` | Export backup (`?scope=user` for own data; full requires admin) |
| POST | `/api/admin/restore` | Restore from JSON backup |
| GET | `/api/admin/users` | List all users |
| DELETE | `/api/admin/users/:id` | Delete user (cannot delete self) |
| GET | `/api/admin/registration` | Get registration open/closed state |
| PUT | `/api/admin/registration` | Toggle registration `{open: bool}` |
| POST | `/api/admin/invitations` | Create invitation `{email}` → sends email |
| GET | `/api/admin/invitations` | List pending invitations |
| DELETE | `/api/admin/invitations/:id` | Revoke invitation |
| GET | `/api/admin/logs` | Log entries. Params: `category`, `user_id`, `search`, `date_from`, `date_to`, `limit`, `offset` |
| GET | `/api/admin/logs/stats` | Log category counts |
| GET | `/api/admin/base-url` | Get base URL setting |
| PUT | `/api/admin/base-url` | Set base URL `{base_url}` |
| GET | `/api/admin/smtp` | Get SMTP config (password masked) |
| PUT | `/api/admin/smtp` | Save SMTP config |
| POST | `/api/admin/smtp/test` | Send test email `{recipient}` |
+79
View File
@@ -165,6 +165,85 @@ Permission resolution is centralised in `services/access.py`. `get_project_permi
`category` (`audit`/`usage`/`error`), `user_id` FK nullable (SET NULL on delete), `username` (denormalised), `action`, `endpoint`, `method`, `status_code`, `duration_ms`, `error_type`, `error_message`, `traceback`, `details` JSONB.
## Detailed File Reference
### Backend (`src/fabledassistant/`)
| File | Responsibility |
|------|---------------|
| `app.py` | Quart app factory; SPA via 404 handler; JSON 404/500 for API; request logging; security headers in `after_request` |
| `auth.py` | `login_required`, `admin_required`, `get_current_user_id` — shared `_check_auth()` helper |
| `config.py` | All config from env vars + Docker secret file support (`_read_secret`); `SECURE_COOKIES`, `TRUST_PROXY_HEADERS`, `OLLAMA_NUM_CTX`; `oidc_enabled()`, `searxng_enabled()` classmethods |
| `rate_limit.py` | In-memory sliding-window rate limiter (`asyncio.Lock` + `defaultdict`); `is_rate_limited(key, max, window)` |
| `models/note.py` | Unified Note model (notes + tasks) |
| `models/user.py` | `id`, `username`, `email`, `password_hash` (nullable — NULL for OAuth-only), `oauth_sub` (unique nullable), `role`, `session_version` |
| `models/conversation.py` | `Conversation` + `Message` models |
| `models/app_log.py` | `AppLog` with `category`, denormalised `username`, `details` JSONB |
| `routes/api.py` | `/api` blueprint; `GET /api/health` (public) |
| `routes/auth.py` | Register, login, logout, me, password/email change, password reset, invite registration, OAuth login+callback; rate limiting; `LOCAL_AUTH_ENABLED` guards |
| `routes/admin.py` | Backup, restore, user management, registration toggle, invitations, base URL, SMTP (admin only) |
| `routes/chat.py` | Conversations CRUD; SSE generation stream; model pull/delete/list/warm; briefing conversation routes |
| `routes/notes.py` | Notes CRUD + wikilinks + backlinks + assist + link suggestions + version history + drafts |
| `routes/tasks.py` | Tasks CRUD; `POST` accepts `project` name string (resolved to `project_id`) |
| `routes/quick_capture.py` | `POST /api/quick-capture` — single-shot natural language item creation |
| `routes/search.py` | `GET /api/search` — semantic + keyword hybrid search |
| `services/auth.py` | `create_user`, `authenticate`, user lookups, password reset tokens, invitation tokens |
| `services/oauth.py` | OIDC discovery (cached), PKCE auth URL, code exchange, `find_or_create_oauth_user` |
| `services/llm.py` | `build_context()`, RAG injection, history summarisation, `stream_chat_with_tools()`, URL fetching, SSRF guard |
| `services/generation_task.py` | `run_generation()` — full chat pipeline: intent routing, tool loop, SSE fan-out, push notification; `run_assist_generation()` |
| `services/intent.py` | `classify_intent()` — fast non-streaming LLM call; intent skip heuristic; `_PRIOR_WORK_REFS` fast-path |
| `services/tools.py` | All LLM tool definitions + `execute_tool()` dispatcher; duplicate guards; `_resolve_project()` 4-step lookup |
| `services/embeddings.py` | `upsert_note_embedding()`, `semantic_search_notes()` (pgvector cosine similarity) |
| `services/generation_buffer.py` | In-memory SSE event buffer; `cancel_event`; 60s cleanup; supports both chat (int keys) and assist (string keys) |
| `services/briefing_pipeline.py` | Two-lane parallel gather → LLM synthesis → `GenerationBuffer` stream |
| `services/briefing_scheduler.py` | APScheduler `BackgroundScheduler`; slots with catch-up logic; async-safe via `asyncio.create_task` |
| `services/research.py` | SearXNG research pipeline: 5 sub-queries → parallel fetch → synthesis; `search_images` for image category |
| `services/caldav.py` | Full CalDAV event/todo lifecycle; synchronous library calls in asyncio executor |
| `services/images.py` | `fetch_and_store_image()` — SHA-256 dedup, content-type validation, 5 MB cap |
| `services/backup.py` | `export_full_backup()`, `export_user_backup()`, `restore_full_backup()` (version 2 with ID maps) |
| `services/push.py` | VAPID key auto-generation; `send_push_notification()` fire-and-forget; 410 Gone cleanup |
| `services/logging.py` | `log_audit`, `log_usage`, `log_error`, `start_log_retention_loop` (hourly cleanup) |
| `services/weather.py` | Nominatim geocoding + Open-Meteo forecast + per-user DB cache |
| `services/rss.py` | feedparser fetch; per-feed DB cache; prune-to-100 items |
| `services/assist.py` | `build_assist_messages()` — section/whole-doc modes; project context injection |
### Frontend (`frontend/src/`)
| File | Responsibility |
|------|---------------|
| `App.vue` | App shell (`100dvh` flex column); global keyboard shortcuts (`g`+key nav, `n/t/c/e///?`); starts status polling + loads settings on mount |
| `api/client.ts` | `ApiError`, `apiGet/Post/Put/Patch/Delete`, `apiSSEStream` (fetch + ReadableStream), auto 401→login redirect |
| `stores/auth.ts` | User, `isAuthenticated`, `isAdmin`, `oauthEnabled`, `localAuthEnabled`, login/register/logout/checkAuth |
| `stores/chat.ts` | Conversation CRUD; `sendMessage()` SSE streaming; `reconnectIfGenerating()`; message queue (localStorage); `streamingStatus` |
| `stores/notes.ts` | CRUD + tag filter; `resolveTitle`; `convertToTask/Note`; `fetchBacklinks`; `fetchAllTags` |
| `stores/tasks.ts` | CRUD + status/priority filter; `patchStatus` |
| `stores/settings.ts` | `assistantName`, `defaultModel`, `installedModels`; `pullModel()`, `deleteModel()` |
| `stores/push.ts` | `isSupported`, `permission`, `isSubscribed`, `subscribe/unsubscribe` |
| `stores/notifications.ts` | `count`, `items`, `fetchCount`, `fetchAll`, `markRead`, `markAll` |
| `views/HomeView.vue` | Chat-first dashboard; 6 task sections (Overdue → Other); 8 recent notes; inline streaming response |
| `views/ChatView.vue` | `/chat` — SSE streaming; context sidebar (Suggested/In Context); note picker; model selector; message queue; bulk delete |
| `views/WorkspaceView.vue` | `/workspace/:id` — 3-panel (tasks/chat/notes); SSE tool-call watcher; conv persisted to localStorage |
| `views/GraphView.vue` | `/graph` — D3 force-directed; tag/note/project-hub nodes; physics panel; peek panel |
| `views/ProjectView.vue` | Kanban grouped by milestone; advance buttons; milestone management |
| `views/SettingsView.vue` | Tabbed settings (11 tabs); tab state in localStorage |
| `views/NoteEditorView.vue` | Tiptap editor; 2-column layout; AI assist panel; diff view; version history; autosave |
| `views/TaskEditorView.vue` | Tiptap editor; 2-column layout; AI assist panel; task log section; sub-tasks |
| `components/ToolCallCard.vue` | Tool call results; `requires_confirmation` inline confirm/deny; direct POST on "Create anyway" |
| `components/ChatMessage.vue` | Message bubble; markdown rendering; tool call cards; "Save as Note" |
| `components/TagInput.vue` | Chip-based tag input; Enter/comma to confirm; autocomplete from `/api/notes/tags` |
| `components/TiptapEditor.vue` | Tiptap wrapper; markdown↔HTML round-trip; selection change emit; WikilinkDecoration; TagDecoration |
| `components/ShareDialog.vue` | `<Teleport>` modal; user tab + group tab; current shares list |
| `components/WorkspaceTaskPanel.vue` | Milestone-grouped task list; detail slide-over |
| `components/WorkspaceNoteEditor.vue` | List ↔ TipTap editor with autosave |
| `composables/useAssist.ts` | AI assist: section parsing, SSE streaming, accept/reject, LCS diff, persistent draft |
| `composables/useAutoSave.ts` | Interval-based autosave (5 min) with dirty/saving guards |
| `composables/useEditorGuards.ts` | Ctrl+S, `beforeunload` warning, route-leave confirm |
| `composables/useTagSuggestions.ts` | `/api/notes/suggest-tags` call + suggestion state |
| `composables/useListKeyboardNavigation.ts` | j/k/Enter list navigation; focus-in-input guard |
| `extensions/WikilinkDecoration.ts` | ProseMirror decoration plugin highlighting `[[wikilinks]]` |
| `extensions/TagDecoration.ts` | ProseMirror decoration plugin highlighting `#tags` |
| `extensions/WikilinkSuggestion.ts` | `@tiptap/suggestion` extension for `[[` autocomplete |
## Key Services
| Service | Responsibility |
+35
View File
@@ -97,6 +97,41 @@ Required secrets (repo → Settings → Secrets → Actions):
- `REGISTRY_USER` — Forgejo username
- `REGISTRY_TOKEN` — Forgejo PAT with `write:packages` scope
## Migration Chain
Current migration sequence (all idempotent raw SQL):
```
0001 create_notes_table
0002 create_tasks_table
0003 task_note_companion (data migration)
0004 merge_tasks_into_notes
0005 add_chat_tables
0006 add_settings_table
0007 add_title_and_updated_at_indexes
0008 add_users_and_user_id
0009 add_message_status
0010 add_app_logs_table
0011 add_password_reset_tokens
0012 add_invitation_tokens
0013 add_tool_calls_to_messages
0014 add_note_embeddings
0015 add_oauth_fields
0016 add_image_cache
0017 add_projects
0018 add_push_subscriptions
0019 add_note_parent_and_project_fields
0020 add_milestones
0021 add_task_logs
0022 add_note_versions_and_drafts
0023 add_tags_to_note_versions
0024 add_session_version
0025 add_sharing_and_notifications
0026 add_briefing_tables
```
**Important:** Do NOT use `op.create_table()` or `sa.Enum()` — SQLAlchemy's event system can fire `CREATE TYPE` even with `create_type=False`, causing failures on re-run. Always use raw SQL with `IF NOT EXISTS` / `DO $$ BEGIN ... EXCEPTION WHEN duplicate_object` guards.
## Project Conventions
### Backend
+59
View File
@@ -117,6 +117,65 @@ Settings are tabbed:
| Logs (admin) | Error, audit, and usage logs with search |
| Groups (admin) | Create/manage groups and membership |
## LLM Chat — Internal Pipeline
### Intent Routing
Before the main model runs, a lightweight intent classifier (`services/intent.py`) runs concurrently with `build_context()`. It makes a fast non-streaming call (~400ms) using a smaller dedicated model (`OLLAMA_INTENT_MODEL`, default `qwen2.5:7b`) to determine if the message requires a tool call.
**Skip heuristic** — Intent classification is skipped entirely for short messages (≤10 words) with no action/object keywords, saving 400800ms on conversational replies.
**Prior-work fast-path** — Phrases like "research you did", "note you made", "using your research" skip the LLM call entirely and route to null (chat), preventing `search_web` from firing when the user references existing notes.
If a tool is detected, the intent's one-sentence `ack` field is streamed as the first chunk (becoming time-to-first-token), then the tool executes, then the main model generates a follow-up response with the tool result.
### Tool Loop
Multi-round tool loop (max 5 rounds). All tool implementations are in `services/tools.py` with `execute_tool()` as the dispatcher.
**Duplicate protection on create_note / create_task:**
1. Exact title match (case-insensitive) → hard block, redirect to `update_note`
2. Fuzzy title match (SequenceMatcher ≥ 82%; punctuation stripped before candidate search) → hard block
3. Semantic content similarity (`semantic_search_notes` threshold 0.90, body ≥ 200 chars) → soft block with `requires_confirmation: true`
**Project resolution** (`_resolve_project`): 4-step lookup — (1) exact DB match, (2) `query in title` substring, (3) `title in query` reverse substring, (4) SequenceMatcher ≥ 0.55.
### Model Status
Three states returned by `GET /api/chat/status`:
- `not_found` — model not installed; orange indicator in header
- `cold` — installed but not loaded in VRAM; yellow pulsing indicator (first request will be slow)
- `loaded` — hot in VRAM; green indicator
After `warmModel()` is called, a 5s polling loop runs for up to 60s until the indicator turns green.
### Context Window
`OLLAMA_NUM_CTX` (default 16384) controls the context window for all generation calls. Intent classification always uses `num_ctx=4096` to reduce VRAM pressure. History summarisation kicks in at 30 messages, keeps 8 recent, summary max 400 tokens.
### Image Search
`search_images` tool (SearXNG `categories=images`) fetches images server-side, stores on disk (SHA-256 dedup, 5 MB cap), and serves from `/api/images/<id>` — the user's browser never contacts the original image host. Requires `SEARXNG_URL`.
### Web Research Pipeline
Triggered by "research X and make a note" or the Research button in ChatView:
1. Intent model generates 5 focused sub-queries as a JSON array
2. All 5 SearXNG queries run in parallel (200ms stagger)
3. Up to 15 unique URLs fetched in parallel
4. Up to 12 sources passed to synthesis LLM (`num_ctx=16384`, `num_predict=8192`)
5. Note created with `tags=["research"]`
SearXNG tip: add the app server IP to `botdetection.ip_lists.pass_ip` in SearXNG `settings.yml` to bypass the rate limiter for backend requests.
## Roadmap
- Calendar/timeline view for tasks with due dates
- Email integration (read/send via IMAP/SMTP tools in chat)
- Session invalidation on user deletion
- Flutter push notifications (requires FCM/APNs — separate from web VAPID)
- Flutter milestone support in project view
## Keyboard Shortcuts
| Key | Action |