809 lines
68 KiB
Markdown
809 lines
68 KiB
Markdown
# Fabled Assistant - Project Context
|
||
|
||
> **Purpose:** This file is the canonical reference for re-initializing Claude Code
|
||
> context on this project. **Update this file after every change session.**
|
||
|
||
## Session Checklist
|
||
> **IMPORTANT:** At the end of every change session, you MUST:
|
||
> 1. **Update this file** (`summary.md`) to reflect all architectural, data model,
|
||
> API, file structure, and roadmap changes made during the session.
|
||
> 2. **Commit all changes** with a descriptive commit message summarizing what was
|
||
> done (e.g., "Merge tasks into notes: single table with task attributes").
|
||
> Include file-level details in the commit body when the change is non-trivial.
|
||
|
||
## Last Updated
|
||
2026-02-28 — Quick-capture endpoint for Android/external clients; bug fixes: create_note list-title crash, AI Assist 400 error, note picker persistent context, intent routing
|
||
|
||
## Project Overview
|
||
Fabled Assistant is a self-hosted note-taking and task-tracking application with
|
||
integrated LLM capabilities. It is designed to run on container infrastructure
|
||
(Docker Swarm) and connect to Ollama or any self-hostable LLM-compatible system
|
||
for AI-assisted features.
|
||
|
||
## Core Architecture
|
||
|
||
### Stack
|
||
| Layer | Technology | Notes |
|
||
|-------------|-----------|-------|
|
||
| Frontend | Vue 3 + TypeScript + Vite + Pinia + Vue Router | SPA served from the same container as the API |
|
||
| Backend/API | Quart (Python 3.12) | Async framework; serves both API and built frontend static files |
|
||
| LLM | Ollama | Or any OpenAI-compatible self-hosted LLM API |
|
||
| Database | PostgreSQL 16 | asyncpg driver, SQLAlchemy 2.0 async ORM, Alembic migrations |
|
||
| Deployment | Docker Compose | Single-container app + separate DB + LLM service |
|
||
|
||
### Key Design Decisions
|
||
- **Single container for frontend + API:** Quart serves the Vue.js production
|
||
build as static files and exposes the REST API under `/api/`.
|
||
- **Quart chosen for familiarity:** The maintainer (bvandeusen) knows Quart well.
|
||
- **SPA routing via 404 handler:** `app.py` uses `@app.errorhandler(404)` (not a
|
||
catch-all route) to serve static files or fall back to `index.html`. API routes
|
||
(`/api/*`) get JSON 404 responses. This avoids a catch-all `/<path:path>` route
|
||
intercepting API GETs.
|
||
- **LLM integration is a separate service:** The app communicates with Ollama (or
|
||
compatible) over HTTP.
|
||
- **First-class tag field:** Tags live in the `tags ARRAY[text]` column and are
|
||
explicitly set by the client — they are NOT auto-extracted from the body text.
|
||
`extract_tags()` in `utils/tags.py` is preserved but no longer called on save.
|
||
`TagDecoration.ts` keeps visual `#tag` highlighting in existing notes that have
|
||
inline tags for cosmetic backward compatibility.
|
||
- **Hierarchical tags:** `#project/webapp` stored as `"project/webapp"`. Filtering
|
||
by `project` matches both `project` and `project/*` children via SQL `unnest` +
|
||
`LIKE` prefix.
|
||
- **Dark-first theming:** CSS custom properties on `:root` (light) and
|
||
`[data-theme="dark"]`, with `prefers-color-scheme` detection defaulting to dark.
|
||
- **Design tokens:** Standardized CSS custom properties for border-radius
|
||
(`--radius-sm: 6px`, `--radius-md: 8px`, `--radius-lg: 12px`), semantic colors
|
||
(`--color-success`, `--color-warning`, `--color-overlay`), and focus ring
|
||
(`--focus-ring`). All components reference tokens instead of hardcoded values.
|
||
- **Unified note/task model:** A task is just a note with task attributes enabled.
|
||
A note has `status IS NOT NULL` ⟹ it's a task. "Convert to task" sets
|
||
`status='todo'`; "convert to note" clears `status`, `priority`, `due_date`.
|
||
No companion notes, no cascade deletes, no bidirectional sync needed.
|
||
- **Wikilinks:** Obsidian-style `[[Title]]` and `[[Title|Display Text]]` in markdown
|
||
bodies. Clicking a wikilink uses `POST /api/notes/resolve-title` to auto-create
|
||
missing notes.
|
||
- **Background generation architecture:** LLM streaming runs in a detached
|
||
`asyncio.Task` that writes into an in-memory `GenerationBuffer`. SSE clients
|
||
tail the buffer and can reconnect mid-stream without data loss. Buffer has
|
||
`cancel_event` for user-initiated stop. Completed buffers are cleaned up after
|
||
60s grace period. Periodic DB flushes every 5s preserve partial content.
|
||
Both chat and AI Assist use this architecture — assist buffers are keyed by
|
||
`"assist:{user_id}"` (string keys) instead of conversation ID (int keys).
|
||
Assist generation has no DB persistence, no title generation, no cancellation.
|
||
- **SSE over WebSockets for LLM streaming:** SSE clients connect via
|
||
`GET /api/chat/conversations/:id/generation/stream` with `Last-Event-ID`
|
||
reconnection support. Frontend uses `fetch()` + `ReadableStream`. Simpler than
|
||
WebSockets; Quart supports async generators natively for SSE.
|
||
- **Context building server-side:** Backend fetches URL content and searches notes —
|
||
frontend sends the message text + optional context note ID + optional `include_note_ids`.
|
||
Keyword extraction uses simple word splitting with stopword filtering (no embeddings).
|
||
`build_context()` returns `(messages, context_meta)` tuple; metadata includes
|
||
auto-found note IDs/titles sent to frontend via SSE `context` event before streaming.
|
||
Multi-word search splits terms into per-word ILIKE with AND logic (not adjacent match).
|
||
Auto-found notes populate the sidebar but are **not** injected into the system prompt
|
||
automatically — users must click `+` in the sidebar to include them (stable system
|
||
prompt prefix enables Ollama KV cache reuse). Explicitly included notes appear as
|
||
`--- Included Notes ---` in the system prompt.
|
||
- **Reverse proxy required for production:** The app does not terminate TLS. A
|
||
reverse proxy (Nginx/Traefik/Caddy) must sit in front of port 5000. Do not
|
||
expose the app directly to the internet.
|
||
- **Registration auto-closes:** After the first user registers (admin), registration
|
||
is closed by default. The admin can toggle it from `/admin/users`. The setting is
|
||
stored as the admin user's `registration_open` setting key.
|
||
- **No blocking long-running operations:** Any potentially slow operation (model
|
||
pulls, LLM calls, URL fetching, etc.) must never block app startup or freeze the
|
||
UI. Backend uses SSE streaming for incremental responses (chat messages and model
|
||
pull progress). Frontend keeps the UI responsive during async operations (loading
|
||
states, streaming indicators, progress percentages). This applies to all future
|
||
features as well.
|
||
- **Auto-pull model on startup:** Non-blocking background task, logs warning on
|
||
failure so the app still starts even if Ollama isn't ready.
|
||
- **Backlinks:** `GET /api/notes/:id/backlinks` searches all note bodies for
|
||
`[[Title]]` patterns referencing the given note. Results include `type: "note"` or
|
||
`type: "task"` based on whether the linking note has `status IS NOT NULL`.
|
||
- **Idempotent raw SQL migrations:** All Alembic migrations use raw SQL with
|
||
`CREATE TABLE IF NOT EXISTS`, `CREATE INDEX IF NOT EXISTS`, and `DO $$ BEGIN
|
||
CREATE TYPE ... EXCEPTION WHEN duplicate_object` to allow safe re-runs and
|
||
fresh database creation.
|
||
- **Session cookie security:** `HttpOnly` and `SameSite=Lax` always set. `Secure`
|
||
flag controlled by `SECURE_COOKIES` env var (enable when behind TLS).
|
||
- **Default SECRET_KEY warning:** App logs a WARNING on startup if the default
|
||
`dev-secret-change-me` key is in use.
|
||
- **Security headers:** Applied in `app.py` `after_request` using `setdefault` so
|
||
SSE/streaming responses that set their own headers aren't clobbered: `X-Content-Type-Options`,
|
||
`X-Frame-Options: DENY`, `Referrer-Policy`, and a restrictive `Content-Security-Policy`.
|
||
- **Rate limiting:** In-memory sliding-window rate limiter (`rate_limit.py`) applied to
|
||
auth endpoints: login (10/60s), register (5/300s), forgot-password (5/300s, silent drop
|
||
to avoid timing info), reset-password (10/60s). Keys are per-IP.
|
||
- **Proxy-aware client IP:** `TRUST_PROXY_HEADERS` env var enables reading real client IP
|
||
from `X-Forwarded-For` / `X-Real-IP` when behind a trusted reverse proxy. Off by default.
|
||
- **Auto-save:** Both note and task editors auto-save every 5 minutes when editing an
|
||
existing dirty record. Silent on error; shows "Auto-saved" toast on success.
|
||
|
||
### High-Level Component Diagram
|
||
```
|
||
┌─────────────────────────────────────────────┐
|
||
│ Docker Compose │
|
||
│ │
|
||
│ ┌──────────────────────┐ ┌────────────┐ │
|
||
│ │ fabledassistant │ │ ollama │ │
|
||
│ │ ┌────────────────┐ │ │ │ │
|
||
│ │ │ Quart Server │ │ │ LLM API │ │
|
||
│ │ │ ┌──────────┐ │ │ │ │ │
|
||
│ │ │ │ Vue SPA │ │ │ └────────────┘ │
|
||
│ │ │ │ (static) │ │ │ ▲ │
|
||
│ │ │ └──────────┘ │ │ │ │
|
||
│ │ │ ┌──────────┐ │ │ HTTP/REST │
|
||
│ │ │ │ /api/* │──┼──┼─────────┘ │
|
||
│ │ │ └──────────┘ │ │ │
|
||
│ │ │ │ │ │ ┌────────────┐ │
|
||
│ │ │ ▼ │ │ │ PostgreSQL │ │
|
||
│ │ │ ┌──────────┐ │ │ │ 16 │ │
|
||
│ │ │ │ asyncpg │──┼──┼──▶ │ │
|
||
│ │ │ └──────────┘ │ │ └────────────┘ │
|
||
│ │ └────────────────┘ │ │
|
||
│ └──────────────────────┘ │
|
||
└─────────────────────────────────────────────┘
|
||
```
|
||
|
||
## Data Model
|
||
|
||
### Users
|
||
- `id` (int PK), `username` (text UNIQUE NOT NULL), `email` (text nullable),
|
||
`password_hash` (text **nullable** — NULL for OAuth-only accounts),
|
||
`oauth_sub` (text UNIQUE nullable — OIDC subject identifier),
|
||
`role` (text NOT NULL DEFAULT 'user'), `created_at` (timestamptz)
|
||
- First registered user auto-assigned `role='admin'`; claims orphaned data
|
||
- Passwords hashed with bcrypt; OAuth-only users have `password_hash = NULL`
|
||
- `to_dict()` excludes `password_hash`; includes `has_password: bool`
|
||
|
||
### Notes (unified — includes tasks)
|
||
- `id` (int PK), `title` (str), `body` (markdown str), `tags` (ARRAY[str]),
|
||
`parent_id` (nullable FK to self), `user_id` (nullable FK to users, CASCADE),
|
||
`status` (nullable str — `todo`/`in_progress`/`done`),
|
||
`priority` (nullable str — `none`/`low`/`medium`/`high`), `due_date` (nullable date),
|
||
`created_at`, `updated_at`
|
||
- **A note is a task when `status IS NOT NULL`** — the `is_task` property checks this
|
||
- Tags are explicitly provided by the client — NOT extracted from body text
|
||
- Supports hierarchical organization via `parent_id`
|
||
- Lookup by exact title via `get_note_by_title()` for wikilink resolution
|
||
- Auto-create via `get_or_create_note_by_title()` for wikilink clicks
|
||
- `to_dict()` returns: `id`, `title`, `body`, `tags`, `parent_id`, `status`,
|
||
`priority`, `due_date`, `is_task`, `created_at`, `updated_at`
|
||
- Indexes: GIN on tags, B-tree on status, B-tree on title
|
||
|
||
### Task ≡ Note with task attributes
|
||
- No separate tasks table. `routes/tasks.py` imports directly from
|
||
`services/notes.py` with `is_task=True` for listing and
|
||
defaults `status='todo'`, `priority='none'` for creation.
|
||
- "Convert to task" = `update_note(id, status='todo', priority='none')`
|
||
- "Convert to note" = `update_note(id, status=None, priority=None, due_date=None)`
|
||
- Task body field is `body` (not `description`) — standardized with notes
|
||
|
||
### Settings
|
||
- Composite PK: `(user_id, key)` — `user_id` FK to users (CASCADE), `key` (text)
|
||
- `value` (text)
|
||
- Per-user key-value store for settings (assistant name, default model, etc.)
|
||
- CRUD via `services/settings.py`: all functions take `user_id` as first parameter
|
||
|
||
### Conversations
|
||
- `id` (int PK), `title` (str), `model` (str), `user_id` (nullable FK to users, CASCADE), `created_at`, `updated_at`
|
||
- Has many messages (cascade delete)
|
||
- Title auto-generated by LLM on first exchange and re-generated every 10th message
|
||
- Index on `updated_at` for list ordering
|
||
- `to_dict()` returns: `id`, `title`, `model`, `message_count`, `created_at`, `updated_at`
|
||
- `list_conversations()` uses a subquery for `message_count` instead of eager-loading all messages
|
||
|
||
### App Logs
|
||
- `id` (int PK), `category` (text NOT NULL — `audit`/`usage`/`error`),
|
||
`user_id` (int FK users ON DELETE SET NULL), `username` (text — denormalized),
|
||
`action` (text), `endpoint` (text), `method` (text), `status_code` (int),
|
||
`duration_ms` (real), `ip_address` (text), `details` (text — JSON-serialized),
|
||
`created_at` (timestamptz DEFAULT now())
|
||
- Single table with `category` field for all log types
|
||
- Denormalized `username` preserves attribution after user deletion
|
||
- `details` stores flexible JSON data (error tracebacks, audit metadata, etc.)
|
||
- Indexes: `category`, `user_id`, `created_at`, composite `(category, created_at DESC)`
|
||
- Automatic retention cleanup via hourly asyncio task (configurable `LOG_RETENTION_DAYS`, default 90)
|
||
|
||
### Invitation Tokens
|
||
- `id` (int PK), `email` (text NOT NULL), `token_hash` (text NOT NULL UNIQUE),
|
||
`invited_by` (int FK users CASCADE), `expires_at` (timestamptz NOT NULL),
|
||
`used` (boolean NOT NULL DEFAULT FALSE), `created_at` (timestamptz)
|
||
- Admin creates invitation → raw token emailed → recipient registers via `/register-invite?token=...`
|
||
- Token stored as SHA256 hash (same pattern as password reset)
|
||
- 7-day expiration, one-time use, previous unused invitations for same email auto-revoked
|
||
- Indexes: `token_hash`, `email`
|
||
|
||
### Messages
|
||
- `id` (int PK), `conversation_id` (FK to conversations, CASCADE), `role` (str: system/user/assistant),
|
||
`content` (str), `status` (str, default `'complete'` — `complete`/`generating`/`error`),
|
||
`context_note_id` (nullable FK to notes, SET NULL), `tool_calls` (nullable JSONB), `created_at`
|
||
- Index on `conversation_id`
|
||
- `context_note_id` tracks which note was attached as context when message was sent
|
||
- `tool_calls` stores an array of tool call records `[{function, arguments, result, status}]` for assistant messages that used tools
|
||
- `to_dict()` returns: `id`, `conversation_id`, `role`, `content`, `status`, `context_note_id`, `tool_calls`, `created_at`
|
||
- `context_note_title` is NOT a DB column — it is synthesised at query time in `get_conversation_route` via a batch `get_notes_by_ids()` lookup and injected into the message dict so the frontend can display the note title without a separate fetch
|
||
|
||
## Project Structure (Current)
|
||
```
|
||
fabledassistant/
|
||
├── summary.md # This file — canonical project context
|
||
├── pyproject.toml # Python project config (deps include caldav, icalendar, httpx)
|
||
├── Dockerfile # Multi-stage build (Node → Python)
|
||
├── .dockerignore # Prevents secrets/node_modules/__pycache__/.env.* leaking into build
|
||
├── docker-compose.yml # Dev compose (app, PostgreSQL, Ollama) — app service has healthcheck
|
||
├── docker-compose.prod.yml # Production stack (Docker Swarm, secrets, network isolation)
|
||
├── alembic.ini # Alembic config (prepend_sys_path = src)
|
||
├── docs/
|
||
│ └── oauth-setup.md # Step-by-step Authentik OIDC setup guide with example docker-compose config
|
||
├── alembic/
|
||
│ ├── env.py # Async migration runner
|
||
│ └── versions/
|
||
│ ├── 0001_create_notes_table.py # Notes table (raw SQL, idempotent)
|
||
│ ├── 0002_create_tasks_table.py # Tasks table + enums (raw SQL, idempotent)
|
||
│ ├── 0003_task_note_companion.py # Data migration: create companion notes for existing tasks
|
||
│ ├── 0004_merge_tasks_into_notes.py # Add task columns to notes, migrate data, drop tasks table
|
||
│ ├── 0005_add_chat_tables.py # Conversations + messages tables with FKs and indexes
|
||
│ ├── 0006_add_settings_table.py # Settings key-value table
|
||
│ ├── 0007_add_title_and_updated_at_indexes.py # B-tree indexes on notes.title and conversations.updated_at
|
||
│ ├── 0008_add_users_and_user_id.py # Users table, user_id FKs on notes/conversations/settings, composite PK for settings
|
||
│ ├── 0009_add_message_status.py # Add status column to messages table (default 'complete')
|
||
│ ├── 0010_add_app_logs_table.py # App logs table for audit, usage, and error logging
|
||
│ ├── 0011_add_password_reset_tokens.py # Password reset tokens table
|
||
│ ├── 0012_add_invitation_tokens.py # Invitation tokens table
|
||
│ ├── 0013_add_tool_calls_to_messages.py # Add tool_calls JSONB column to messages
|
||
│ ├── 0014_add_note_embeddings.py # note_embeddings table for semantic search (note_id PK, user_id, embedding JSONB, updated_at)
|
||
│ └── 0015_add_oauth_fields.py # Add oauth_sub UNIQUE column; DROP NOT NULL on password_hash
|
||
├── src/
|
||
│ └── fabledassistant/
|
||
│ ├── __init__.py
|
||
│ ├── app.py # Quart app factory: SPA via 404 handler, JSON 404/500 for API, request logging, security headers (after_request)
|
||
│ ├── auth.py # Auth decorators: login_required, admin_required, get_current_user_id — shared _check_auth() helper
|
||
│ ├── config.py # Config from env vars + Docker secrets file support (_read_secret) + SECURE_COOKIES + TRUST_PROXY_HEADERS + OLLAMA_NUM_CTX (KV cache window, default 8192) + OIDC_ISSUER/CLIENT_ID/CLIENT_SECRET/SCOPES + LOCAL_AUTH_ENABLED + oidc_enabled() classmethod + SEARXNG_URL + searxng_enabled() classmethod
|
||
│ ├── rate_limit.py # In-memory sliding-window rate limiter (asyncio.Lock + defaultdict); is_rate_limited(key, max, window)
|
||
│ ├── models/
|
||
│ │ ├── __init__.py # async_session factory, Base, imports all models
|
||
│ │ ├── user.py # User model (id, username, email, password_hash nullable, oauth_sub unique nullable, role, created_at); to_dict() includes has_password bool
|
||
│ │ ├── note.py # Note model (unified: id, title, body, tags[], parent_id, user_id, status, priority, due_date, timestamps)
|
||
│ │ ├── conversation.py # Conversation + Message models with user_id
|
||
│ │ ├── setting.py # Setting model (composite PK: user_id + key, value TEXT)
|
||
│ │ ├── app_log.py # AppLog model (id, category, user_id, username, action, endpoint, method, status_code, duration_ms, ip_address, details, created_at)
|
||
│ │ └── invitation.py # InvitationToken model (id, email, token_hash, invited_by, expires_at, used, created_at)
|
||
│ ├── routes/
|
||
│ │ ├── __init__.py
|
||
│ │ ├── api.py # /api blueprint with /health endpoint (public)
|
||
│ │ ├── auth.py # /api/auth blueprint: register, login, logout, me, password, email change, status, password reset, invitation registration, OAuth login+callback — rate limiting, LOCAL_AUTH_ENABLED guards, _client_ip() helper
|
||
│ │ ├── admin.py # /api/admin blueprint: backup, restore, user management, registration toggle, invitations, base URL, SMTP (admin only)
|
||
│ │ ├── chat.py # /api/chat blueprint: conversations CRUD, SSE message streaming (all @login_required)
|
||
│ │ ├── notes.py # /api/notes CRUD + wikilinks + backlinks + tag suggestions (all @login_required)
|
||
│ │ ├── quick_capture.py # /api/quick-capture POST — mobile/external single-shot item creation (session auth)
|
||
│ │ ├── tasks.py # /api/tasks CRUD + PATCH status (all @login_required)
|
||
│ │ └── settings.py # /api/settings GET/PUT, GET /models — per-user settings + installed model list (@login_required)
|
||
│ ├── services/
|
||
│ │ ├── auth.py # Auth business logic: hash_password, verify_password, create_user (password optional, oauth_sub kwarg), authenticate (returns None for OAuth-only users), get_user_by_id/username/email/oauth_sub, update_user_email, link_oauth_sub, is_registration_open, list_users, delete_user, password reset tokens, invitation tokens
|
||
│ │ ├── oauth.py # OIDC/OAuth2 service: get_oidc_config (discovery, cached), build_auth_url (PKCE), exchange_code, get_userinfo, find_or_create_oauth_user (sub lookup → email auto-link → create)
|
||
│ │ ├── backup.py # Backup/restore: export_full_backup, export_user_backup, restore_full_backup
|
||
│ │ ├── notes.py # CRUD with user_id isolation, is_task filter, convert, backlinks, search_notes_for_context
|
||
│ │ ├── llm.py # Ollama interaction: build_context with user_id, streaming (stream_chat + stream_chat_with_tools), ChatChunk dataclass, URL fetching; generate_completion accepts num_ctx kwarg; uses Config.OLLAMA_NUM_CTX for KV cache window
|
||
│ │ ├── chat.py # Conversation CRUD with user_id isolation, add_message, save/summarize as note (LLM-titled, chat-tagged)
|
||
│ │ ├── generation_buffer.py # In-memory SSE event buffer with cancel_event, reconnect support, auto-cleanup; supports chat (int keys) and assist (string keys)
|
||
│ │ ├── generation_task.py # Background asyncio tasks: run_generation (chat, DB flush, titles, intent-first pipeline + tool loop) + run_assist_generation (lightweight, no DB); _INTENT_TRIGGER_WORDS + _should_skip_intent() for skipping intent on conversational messages
|
||
│ │ ├── intent.py # Intent routing: classify_intent() makes fast non-streaming LLM call (num_ctx=4096); IntentResult has ack field (one-sentence acknowledgment streamed as TTFT)
|
||
│ │ ├── tools.py # LLM tool definitions (create/delete note+task, update_note w/tag management, get_note, list_notes, search_notes w/type filter, list_tasks, full CalDAV suite incl. search_todos, search_web, research_topic when SearXNG enabled) + execute_tool dispatcher
|
||
│ │ ├── research.py # SearXNG research pipeline: parallel query/fetch, streaming synthesis; run_research_pipeline() → Note; constants SEARXNG_QUERIES=5, PAGES_PER_QUERY=3, MAX_SYNTHESIS_SOURCES=12, CHARS_PER_SOURCE=2000
|
||
│ │ ├── tag_suggestions.py # LLM-powered tag suggestions: suggest_tags() builds prompt with existing tags, calls generate_completion, parses JSON response
|
||
│ │ ├── caldav.py # CalDAV integration: full event lifecycle (create/list/search/update/delete), todos (create/list/search/update/complete/delete), list_calendars, timezone (ZoneInfo), reminders (VALARM), attendees, multi-calendar search
|
||
│ │ ├── settings.py # Settings CRUD with user_id isolation: get_setting, set_setting, set_settings_batch, get_all_settings
|
||
│ │ ├── logging.py # App logging: log_audit, log_usage, log_error, get_logs, get_log_stats, delete_old_logs, start_log_retention_loop
|
||
│ │ ├── email.py # SMTP email service: get_smtp_config, is_smtp_configured, send_email, send_test_email
|
||
│ │ └── notifications.py # Notification service: notify_security_event, check_due_tasks, start_notification_loop
|
||
│ ├── utils/
|
||
│ │ ├── __init__.py
|
||
│ │ └── tags.py # extract_tags() — regex #tag extraction, skips code fences
|
||
│ └── static/ # Vue production build (generated by Dockerfile)
|
||
└── frontend/
|
||
├── package.json # deps: vue, pinia, vue-router, marked, dompurify
|
||
├── vite.config.ts
|
||
├── tsconfig.json
|
||
├── src/
|
||
│ ├── App.vue # Shell: .app-shell (100dvh flex column) with AppHeader + .app-content (flex:1, scrollable) wrapping router-view; starts status polling + loads settings on mount
|
||
│ ├── main.ts # App init, imports theme.css
|
||
│ ├── assets/
|
||
│ │ ├── theme.css # CSS custom properties: light/dark themes, body reset, design tokens, responsive breakpoints (480/768/1024), utility classes (.hide-mobile/.hide-desktop), mobile touch targets; light mode uses indigo brand color (#6366f1) + indigo-tinted off-white backgrounds (#f5f5fb/#ededf5)
|
||
│ │ └── editor-shared.css # Shared editor styles for NoteEditorView + TaskEditorView: layout, toolbar, tag suggestions, assist panel, diff view, modal, responsive, .inline-assist-btn (global — required for teleported elements)
|
||
│ ├── api/
|
||
│ │ └── client.ts # ApiError class, apiGet/apiPost/apiPut/apiPatch/apiDelete + apiSSEStream (fetch+ReadableStream), apiStreamPost (legacy), auto 401→login redirect
|
||
│ ├── composables/
|
||
│ │ ├── useTheme.ts # Theme toggle, localStorage, prefers-color-scheme
|
||
│ │ ├── useShortcuts.ts # Shared showShortcuts ref + open/close/toggle helpers (used by App.vue + AppHeader.vue)
|
||
│ │ ├── useAutocomplete.ts # Legacy textarea autocomplete (replaced by Tiptap suggestion extensions)
|
||
│ │ └── useAssist.ts # AI Assist composable: section parsing, target selection, two-step POST+SSE streaming (apiPost → apiSSEStream), accept/reject (accept() resets to idle on doc-change error), proofread (full-document), LCS line diff (DiffLine/computeDiff), isProofreading ref; watches body ref for auto-sync
|
||
│ ├── stores/
|
||
│ │ ├── auth.ts # Auth state: user, isAuthenticated, isAdmin, oauthEnabled, localAuthEnabled, login/register/logout/checkAuth/checkHasUsers
|
||
│ │ ├── notes.ts # CRUD + tag filter, resolveTitle, convertToTask, convertToNote, fetchBacklinks, fetchAllTags (with toast errors)
|
||
│ │ ├── tasks.ts # CRUD + status/priority filter, patchStatus (with toast errors)
|
||
│ │ ├── chat.ts # Conversation CRUD, sendMessage (SSE streaming, includeNoteIds param), status polling (memory-leak-safe _pollUntilLoaded), running models, model warming, updateConversationModel (with toast errors)
|
||
│ │ ├── settings.ts # App settings: assistantName, defaultModel, installedModels, defaultChatModel, defaultIntentModel, pullModel, deleteModel (with toast errors)
|
||
│ │ └── toast.ts # Toast notification state (success/error/warning), 4s auto-dismiss, dismiss(id)
|
||
│ ├── types/
|
||
│ │ ├── auth.ts # User interface (incl. has_password bool), AuthStatus interface (incl. oauth_enabled, local_auth_enabled)
|
||
│ │ ├── note.ts # Note interface (with status, priority, due_date, is_task) + TaskStatus, TaskPriority types + NoteListResponse
|
||
│ │ ├── chat.ts # ToolCallRecord, Message, Conversation, ConversationDetail, ContextMeta, OllamaModel, RunningModel, OllamaStatus interfaces
|
||
│ │ ├── settings.ts # AppSettings interface, ModelInfo interface (name, description, size, bestFor, category)
|
||
│ │ └── task.ts # Task = re-export of Note; TaskListResponse
|
||
│ ├── extensions/
|
||
│ │ ├── TagDecoration.ts # ProseMirror decoration plugin: highlights #tags with .inline-tag class
|
||
│ │ ├── WikilinkDecoration.ts # ProseMirror decoration plugin: highlights [[wikilinks]] with .wikilink class
|
||
│ │ ├── TagSuggestion.ts # @tiptap/suggestion extension for #tag autocomplete (suppressed at line start for headings)
|
||
│ │ ├── WikilinkSuggestion.ts # @tiptap/suggestion extension for [[wikilink]] autocomplete
|
||
│ │ └── suggestionRenderer.ts # Shared Vue renderer for suggestion dropdowns (creates/positions SuggestionDropdown)
|
||
│ ├── utils/
|
||
│ │ ├── tags.ts # extractTags(), linkifyTags() (with (?<!&) to skip HTML entities), linkifyWikilinks()
|
||
│ │ ├── markdown.ts # renderMarkdown() (full) + renderPreview() (strips links/images) — decodes HTML entities before marked, replaces ' after sanitization; explicit DOMPurify config with FORCE_BODY
|
||
│ │ ├── markdownSerializer.ts # Tiptap JSON → markdown serializer: handles all StarterKit nodes + marks
|
||
│ │ └── sectionParser.ts # parseMarkdownSections() (heading-based) + parseFallbackSections() (paragraph/Q&A-style — single-line ≤120 char paragraphs become pseudo-headings; top-level bullet/numbered list items become individual sections)
|
||
│ ├── views/
|
||
│ │ ├── LoginView.vue # Login form; "Login with Authentik" SSO button when oauth_enabled; hides password form when local_auth_disabled; handles ?error=oauth query param
|
||
│ │ ├── RegisterView.vue # Register form with password confirmation; shows "closed" message when registration disabled
|
||
│ │ ├── RegisterInviteView.vue # Invitation-based registration: validates token, creates account with pre-set email
|
||
│ │ ├── UserManagementView.vue # Admin user management: registration toggle, invitations (send/revoke), user list with delete
|
||
│ │ ├── ChatView.vue # Dedicated /chat page: responsive sidebar (overlay on mobile), bubble messages, note picker, context sidebar with “In Context” (user-included, ×) + “Suggested” (auto-found, +), model selector in header
|
||
│ │ ├── HomeView.vue # Chat-first dashboard: quick actions + chat widget (top, full-width), inline response panel, two-column grid (3fr tasks / 2fr notes); task sections: Overdue, Due Today, Due This Week, High Priority, In Progress, Other (capped 10, due-dated first); 8 recent notes; model warming on mount
|
||
│ │ ├── SettingsView.vue # Settings page: assistant name, chat/intent model dropdowns (populated from installed models), email change (with password confirmation for local-auth users), change password, notifications, CalDAV, SMTP (admin), base URL (admin), data export/restore (admin)
|
||
│ │ ├── NotesListView.vue # Note list: search, sort, tag filter pills, pagination
|
||
│ │ ├── NoteEditorView.vue # Create/edit: Tiptap editor, sticky toolbar, TagInput chip field (between title and body), AI Assist panel (right-side 320px, collapsible), line-level diff view, Proofread action, floating inline ✨ pill on text selection, LLM tag suggestions (adds chips), Ctrl+S, auto-save (5min), unsaved guard; styles from editor-shared.css
|
||
│ │ ├── NoteViewerView.vue # Markdown render, wikilink auto-create, convert-to-task, backlinks, table of contents sidebar
|
||
│ │ ├── TasksListView.vue # Task list: search, status/priority filters, sort, pagination
|
||
│ │ ├── TaskEditorView.vue # Create/edit task: Tiptap editor, sticky toolbar, TagInput chip field (between metadata fields and body), AI Assist panel (right-side 320px, collapsible), line-level diff view, Proofread action, floating inline ✨ pill on text selection, LLM tag suggestions (adds chips), Ctrl+S, auto-save (5min), dirty guard; styles from editor-shared.css + task-specific scoped styles
|
||
│ │ └── TaskViewerView.vue # Task detail: rendered markdown, badges (isOverdue uses ISO string comparison), convert-to-note, backlinks, table of contents sidebar
|
||
│ ├── components/
|
||
│ │ ├── LogsView.vue # Admin log viewer: stats summary, category/search/date filters, paginated table with IP column + expandable detail rows (expands on ip_address or details)
|
||
│ │ ├── AppHeader.vue # Nav bar: brand, nav links (incl. admin Logs), status indicator, theme toggle, user info + logout, hamburger menu (mobile)
|
||
│ │ ├── ChatPanel.vue # Slide-out chat panel (right side overlay, receives contextNoteId prop), bubble-style messages, floating dark input, note picker, context pills with promote (+) only (no exclude)
|
||
│ │ ├── ModelSelector.vue # Model dropdown (v-model pattern): fetches installed + running models, hot/cold indicators
|
||
│ │ ├── DashboardChatInput.vue # Inline chat bar: ModelSelector + note picker + textarea + send button; emits submit event
|
||
│ │ ├── ToolCallCard.vue # Compact card for tool call results (created/deleted task/note, note content, notes list, search results, CalDAV events/todos, errors) + suggested tag pills with apply-on-click
|
||
│ │ ├── ChatMessage.vue # Message bubble: markdown rendering, tool call cards, configurable assistant name label, "Save as Note" action on assistant messages
|
||
│ │ ├── NoteCard.vue # Card with rendered markdown preview (v-html), TagPill, tag-click emit, hover edit button
|
||
│ │ ├── TaskCard.vue # Card with rendered preview (body not description), StatusBadge (clickable), PriorityBadge, due date, tags, hover edit button
|
||
│ │ ├── StatusBadge.vue # Color-coded status badge, optional clickable cycling
|
||
│ │ ├── PriorityBadge.vue # Color-coded priority indicator (hidden for "none")
|
||
│ │ ├── MarkdownToolbar.vue # Tiptap command-based toolbar: bold/italic/link/list/heading with active state highlighting
|
||
│ │ ├── TagInput.vue # Chip-based tag input: Enter/comma/click to confirm, Backspace removes last, × removes chip, autocomplete from /api/notes/tags, space→hyphen sanitization
|
||
│ │ ├── TiptapEditor.vue # Tiptap wrapper: markdown↔HTML round-trip, paste handling, selection change emit (closest-match offset strategy), expose editor; no fetchTags prop (TagSuggestion removed, TagDecoration kept)
|
||
│ │ ├── SuggestionDropdown.vue # Shared autocomplete dropdown for tag/wikilink suggestions
|
||
│ │ ├── SearchBar.vue # Debounced search input
|
||
│ │ ├── TagPill.vue # Clickable/dismissible tag pill
|
||
│ │ ├── PaginationBar.vue # Prev/next + page numbers
|
||
│ │ ├── TableOfContents.vue # Sticky sidebar TOC: parses markdown headings, smooth-scroll on click, hidden ≤1200px
|
||
│ │ └── ToastNotification.vue # Fixed-position toast container with close button, warning support
|
||
│ └── router/
|
||
│ └── index.ts # Routes: /, /login, /register, /register-invite, /notes/*, /tasks/*, /chat, /chat/:id, /settings, /admin/users, /admin/logs; beforeEach auth guard
|
||
└── public/
|
||
```
|
||
|
||
## API Endpoints (Current)
|
||
|
||
| Method | Path | Description |
|
||
|--------|------|-------------|
|
||
| GET | `/api/health` | Health check (public) |
|
||
| GET | `/api/auth/status` | Returns `{has_users, registration_open, oauth_enabled, local_auth_enabled}` (public) |
|
||
| POST | `/api/auth/register` | Register new user (first user becomes admin; 403 if registration closed or local auth disabled) |
|
||
| POST | `/api/auth/login` | Login with username/password (403 if local auth disabled; returns None for OAuth-only users) |
|
||
| POST | `/api/auth/logout` | Logout (clear session) |
|
||
| GET | `/api/auth/me` | Get current user info (includes `has_password` bool) |
|
||
| PUT | `/api/auth/password` | Change password (body: `{current_password, new_password}`) |
|
||
| PUT | `/api/auth/email` | Change email (body: `{email, password?}`; password required only if user has local password) |
|
||
| GET | `/api/auth/oauth/login` | Initiate OIDC flow — generates PKCE verifier + state, stores in session, redirects to provider |
|
||
| GET | `/api/auth/oauth/callback` | OIDC callback — exchanges code, fetches userinfo, finds/creates user, sets session, redirects to `/` |
|
||
| GET | `/api/admin/backup` | Export backup (`?scope=user` for own data, full requires admin) |
|
||
| POST | `/api/admin/restore` | Restore from JSON backup (admin only) |
|
||
| GET | `/api/admin/users` | List all users (admin only) |
|
||
| DELETE | `/api/admin/users/:id` | Delete a user (admin only, cannot delete self) |
|
||
| GET | `/api/admin/registration` | Get registration open/closed status (admin only) |
|
||
| PUT | `/api/admin/registration` | Toggle registration open/closed (admin only, body: `{open: bool}`) |
|
||
| GET | `/api/admin/logs` | List log entries (admin only, params: `category`, `user_id`, `search`, `date_from`, `date_to`, `limit`, `offset`) |
|
||
| GET | `/api/admin/logs/stats` | Get log category counts (admin only) |
|
||
| GET | `/api/notes` | List notes (params: `q`, `tag`, `sort`, `order`, `limit`, `offset`; defaults to `is_task=false` — plain notes only; `?is_task=true` for tasks, `?all=true` for everything) |
|
||
| POST | `/api/notes` | Create note (body: `{title, body, tags?: string[], status?, priority?, due_date?}` — tags explicit array) |
|
||
| GET | `/api/notes/tags` | List all tags from notes table (param: `q` for filter) |
|
||
| POST | `/api/notes/suggest-tags` | LLM-suggest tags (body: `{title, body, current_tags?: string[]}`, returns `{suggested_tags: [...]}`) |
|
||
| POST | `/api/notes/:id/append-tag` | Add tag to note's tags array with deduplication (body: `{tag}`, returns updated note) |
|
||
| GET | `/api/notes/by-title?title=...` | Resolve note by exact title (case-insensitive) |
|
||
| POST | `/api/notes/resolve-title` | Get-or-create note by title (for wikilink clicks) |
|
||
| GET | `/api/notes/:id` | Get single note (response includes `is_task`, `status`, `priority`, `due_date`) |
|
||
| PUT | `/api/notes/:id` | Update note (body: `{title?, body?, tags?: string[], status?, priority?, due_date?}` — omitting tags keeps existing) |
|
||
| DELETE | `/api/notes/:id` | Delete note (simple delete, no cascade) |
|
||
| POST | `/api/notes/:id/convert-to-task` | Set `status='todo'`, `priority='none'` on note (returns 200) |
|
||
| POST | `/api/notes/:id/convert-to-note` | Clear `status`, `priority`, `due_date` from note (returns 200) |
|
||
| GET | `/api/notes/:id/backlinks` | List notes/tasks that reference this note via wikilinks |
|
||
| POST | `/api/notes/assist` | Launch background assist generation, return 202 (body: `{body, target_section, instruction}`; 409 if already running) |
|
||
| GET | `/api/notes/assist/stream` | SSE endpoint tailing assist generation buffer; supports `Last-Event-ID` reconnection; emits `chunk`, `done`, `error` events with 15s keepalives |
|
||
| GET | `/api/auth/invitation/:token` | Validate invitation token, returns `{valid, email?}` (public) |
|
||
| POST | `/api/auth/register-with-invite` | Register with invitation token (body: `{token, username, password}`) (public) |
|
||
| GET | `/api/admin/base-url` | Get application base URL setting (admin only) |
|
||
| PUT | `/api/admin/base-url` | Set application base URL (admin only, body: `{base_url}`) |
|
||
| POST | `/api/admin/invitations` | Create invitation (admin only, body: `{email}`) — sends email with registration link |
|
||
| GET | `/api/admin/invitations` | List pending invitations (admin only) |
|
||
| DELETE | `/api/admin/invitations/:id` | Revoke invitation (admin only) |
|
||
| GET | `/api/tasks` | List tasks (params: `q`, `tag`, `status`, `priority`, `due_before`, `due_after`, `sort`, `order`, `limit`, `offset`) — queries notes where `status IS NOT NULL` |
|
||
| POST | `/api/tasks` | Create task (body: `{title, body, tags?: string[], status?, priority?, due_date?}` — accepts `description` as fallback for `body`) |
|
||
| GET | `/api/tasks/:id` | Get single task |
|
||
| PUT | `/api/tasks/:id` | Update task (body: `{title?, body?, tags?: string[], status?, priority?, due_date?}` — accepts `description` as fallback for `body`) |
|
||
| PATCH | `/api/tasks/:id/status` | Quick status toggle (body: `{status}`) |
|
||
| DELETE | `/api/tasks/:id` | Delete task (simple delete) |
|
||
| GET | `/api/chat/conversations` | List conversations (params: `limit`, `offset`) |
|
||
| POST | `/api/chat/conversations` | Create conversation (body: `{title?, model?}`) |
|
||
| GET | `/api/chat/conversations/:id` | Get conversation with all messages |
|
||
| DELETE | `/api/chat/conversations/:id` | Delete conversation (cascades to messages) |
|
||
| PATCH | `/api/chat/conversations/:id` | Update conversation title or model (body: `{title?, model?}`) |
|
||
| POST | `/api/chat/conversations/:id/messages` | Start generation: save user message, launch background task, return 202 (body: `{content, context_note_id?, include_note_ids?}`) |
|
||
| GET | `/api/chat/conversations/:id/generation/stream` | SSE endpoint tailing generation buffer; supports `Last-Event-ID` reconnection; emits `context`, `chunk`, `done`, `error` events |
|
||
| POST | `/api/chat/conversations/:id/generation/cancel` | Cancel active generation (sets cancel_event, saves partial content) |
|
||
| POST | `/api/chat/messages/:id/save-as-note` | Save assistant message as a new note (LLM-generated title, tagged `chat`) |
|
||
| POST | `/api/chat/conversations/:id/summarize` | Summarize conversation via LLM, save as note |
|
||
| GET | `/api/chat/ps` | List currently loaded (hot) Ollama models (`{models: [{name, size, size_vram, expires_at}]}`) |
|
||
| POST | `/api/chat/warm` | Pre-load a model into Ollama memory (body: `{model}`, returns 202) |
|
||
| GET | `/api/chat/status` | Check Ollama availability and model readiness (`{ollama, model, default_model}`) |
|
||
| GET | `/api/chat/models` | List available Ollama models |
|
||
| POST | `/api/chat/models/pull` | Pull/download a model from Ollama via SSE streaming progress (body: `{model}`, response: SSE with `{status, completed, total}` events) |
|
||
| POST | `/api/chat/models/delete` | Delete a model from Ollama (body: `{model}`) |
|
||
| GET | `/api/admin/smtp` | Get SMTP config (password masked) (admin only) |
|
||
| PUT | `/api/admin/smtp` | Save SMTP config to admin settings (admin only, body: `{smtp_host, smtp_port, ...}`) |
|
||
| POST | `/api/admin/smtp/test` | Send test email (admin only, body: `{recipient}`) |
|
||
| GET | `/api/settings` | Get all app settings as `{key: value}` dict |
|
||
| PUT | `/api/settings` | Update settings (body: `{key: value, ...}`) |
|
||
| GET | `/api/settings/models` | Get installed Ollama models + configured defaults (`{models, default_chat_model, default_intent_model}`) |
|
||
|
||
## Alembic Migrations
|
||
|
||
### Overview
|
||
Migrations use raw SQL for full idempotency — safe to run on a fresh database
|
||
or re-run on an existing one. The Dockerfile runs migrations automatically on
|
||
container startup.
|
||
|
||
### Migration Chain
|
||
```
|
||
0001 → 0002 → 0003 → 0004 → 0005 → 0006 → 0007 → 0008 → 0009 → 0010 → 0011 → 0012 → 0013 → 0014 → 0015
|
||
```
|
||
|
||
### How Migrations Run
|
||
The Dockerfile CMD runs:
|
||
```sh
|
||
(alembic stamp --purge base 2>/dev/null || true) && alembic upgrade head && hypercorn ...
|
||
```
|
||
- `alembic stamp --purge base` clears any stale revision stamps (safe no-op if none)
|
||
- `alembic upgrade head` runs all pending migrations in order
|
||
- Migrations are idempotent so re-running is safe
|
||
|
||
### Creating a New Migration
|
||
When adding a new migration, follow these conventions:
|
||
|
||
1. **Create the migration file:**
|
||
```
|
||
alembic/versions/0005_description.py
|
||
```
|
||
|
||
2. **Use raw SQL for idempotency:**
|
||
```python
|
||
from alembic import op
|
||
|
||
revision = "0005"
|
||
down_revision = "0004"
|
||
|
||
def upgrade() -> None:
|
||
# For new enums:
|
||
op.execute("""
|
||
DO $$ BEGIN
|
||
CREATE TYPE my_enum AS ENUM ('a', 'b');
|
||
EXCEPTION WHEN duplicate_object THEN null;
|
||
END $$
|
||
""")
|
||
# For new tables:
|
||
op.execute("""
|
||
CREATE TABLE IF NOT EXISTS my_table (...)
|
||
""")
|
||
# For new indexes:
|
||
op.execute("CREATE INDEX IF NOT EXISTS ix_name ON table (col)")
|
||
# For adding columns:
|
||
op.execute("""
|
||
DO $$ BEGIN
|
||
ALTER TABLE my_table ADD COLUMN new_col TEXT DEFAULT '';
|
||
EXCEPTION WHEN duplicate_column THEN null;
|
||
END $$
|
||
""")
|
||
|
||
def downgrade() -> None:
|
||
op.execute("DROP INDEX IF EXISTS ix_name")
|
||
op.execute("DROP TABLE IF EXISTS my_table")
|
||
op.execute("DROP TYPE IF EXISTS my_enum")
|
||
```
|
||
|
||
3. **Rebuild and test:**
|
||
```bash
|
||
docker compose up --build
|
||
```
|
||
To test from a completely fresh database:
|
||
```bash
|
||
docker compose down -v && docker compose up --build
|
||
```
|
||
|
||
### Important Notes
|
||
- Do NOT use `op.create_table()` or `sa.Enum()` — SQLAlchemy's event system
|
||
can fire `CREATE TYPE` even with `create_type=False`, causing failures on re-run
|
||
- Always write raw SQL with PostgreSQL idempotency guards
|
||
- Set `down_revision` to chain from the previous migration's `revision`
|
||
- Data migrations (like 0003) should use `op.get_bind()` + `sa.text()` for queries
|
||
|
||
## Implemented Features
|
||
|
||
### Notes & Tasks
|
||
- Full CRUD with markdown bodies, first-class tag array (chip input in editors), hierarchical tag filtering
|
||
- Unified data model: a task is a note with `status IS NOT NULL` — convert freely between note ↔ task
|
||
- Task attributes: status (todo/in_progress/done), priority (none/low/medium/high), due_date
|
||
- Obsidian-style `[[wikilinks]]` with auto-create on click, backlinks ("what links here")
|
||
- Tiptap WYSIWYG editor with markdown round-trip, tag/wikilink autocomplete and decorations, sticky toolbar
|
||
- AI Assist panel (right-side 320px, toggle persisted to localStorage): section targeting, floating ✨ pill on text selection, full-document Proofread action, line-level diff view in review state (LCS algorithm), "Show full text" toggle, accept/reject; panel collapses to bottom 45% on mobile (≤768px)
|
||
- Table of contents sidebar on note/task viewers (auto-generated from headings, hidden ≤1200px)
|
||
- Inline edit buttons on NoteCard/TaskCard (hover on desktop, always visible on touch)
|
||
- Search (ILIKE), sort, tag filter pills, pagination on list views
|
||
|
||
### Dashboard
|
||
- **Chat-first layout:** Quick action chips + chat input at top (full width); no page title.
|
||
Inline streaming response (full width) appears between widget and content grid when active.
|
||
- **Two-column grid:** Tasks left (3fr) / Recent Notes right (2fr); collapses to single column on mobile. Max-width 1400px.
|
||
- **Left column — 6 task sections** in urgency order, all cascading-deduplicated:
|
||
Overdue (red border), Due Today, Due This Week, High Priority (amber border), In Progress, Other (capped at 10).
|
||
"Other" = broad fetch of all non-done tasks deduped against shown sections; due-dated items first (asc), then undated by priority desc.
|
||
- **Right column** — 8 most recently edited notes.
|
||
- Priority-aware sorting (priority desc → due date asc) within each section.
|
||
- All sections hidden when empty; marking a task done removes it from all lists.
|
||
- **Inline chat:** Streams response inline — no navigation to `/chat`. Tool calls shown live.
|
||
Conversational (no tool calls) response promotes "Continue this conversation →" button.
|
||
- **Quick action chips:** Pre-defined prompts above the chat input.
|
||
- **Auto-focus:** Dashboard chat input gains focus on page mount.
|
||
- **Keyboard shortcut — Escape in ChatView:** Cascade close: note picker → mobile sidebar → clear textarea → navigate to `/` home.
|
||
- **Keyboard shortcuts overlay:** Press `?` or click `?` in nav bar. State shared via `useShortcuts` composable.
|
||
|
||
### LLM Chat
|
||
- Ollama integration via async HTTP (httpx), default model `qwen3:latest` (better tool support than mistral), auto-pull on startup
|
||
- Background generation with `GenerationBuffer` (in-memory SSE fan-out, `Last-Event-ID` reconnect, 60s cleanup)
|
||
- Stop generation with partial content preservation
|
||
- Note-aware context building: current note + keyword search for related notes + URL fetching
|
||
- **Context sidebar (Phase 21):** Right panel in ChatView shows two sections: **Suggested** (auto-found
|
||
by semantic/keyword search — click `+` to include) and **In Context** (explicitly included by user —
|
||
click `×` to remove). Removing an included note moves it back to Suggested. Hidden on mobile (≤768px).
|
||
Auto-found notes are shown as sidebar candidates only — they are NOT injected into the system prompt
|
||
automatically, keeping the system prompt prefix stable for Ollama KV cache reuse.
|
||
- Note picker (paperclip) in chat input adds notes directly to **persistent** `includedNotes` (not a
|
||
one-shot attachment) — picked notes appear in "In Context" and remain for the entire conversation.
|
||
`context_note_title` synthesised server-side at conversation load via batch `get_notes_by_ids()`;
|
||
message badge shows note title instead of "Note #N".
|
||
- LLM-generated conversation titles (re-generated every 10th message)
|
||
- Save assistant messages as notes (LLM-titled, chat-tagged); summarize conversations
|
||
- Dedicated `/chat` page with responsive sidebar + slide-out chat panel from header
|
||
- Model catalog with installed/available tabs, download progress, select, remove
|
||
- Per-conversation model selection via ModelSelector dropdown in chat header (persisted via PATCH)
|
||
- Dashboard inline chat input: model selector + note picker + textarea; creates conversation and navigates
|
||
- Model warming: default model pre-loaded into Ollama on dashboard mount via fire-and-forget POST
|
||
- **Model load state indicator:** `GET /api/chat/status` now queries `/api/tags` and `/api/ps` in
|
||
parallel. Model status has three values: `"not_found"` (not installed), `"cold"` (installed but
|
||
not in VRAM — first response will be slow), `"loaded"` (hot in VRAM, fast response).
|
||
`chatReady` is true for both `"cold"` and `"loaded"` since cold models still work.
|
||
AppHeader shows five distinct visual states: gray pulse (checking), red (Ollama down), orange
|
||
(model not installed), yellow pulse (cold), green (loaded). Each state has a short inline text
|
||
label ("Cold", "Ready", "Offline", etc.) visible without hovering, plus a tooltip with detail.
|
||
After `warmModel()` is called, a fast 5s polling loop runs for up to 60s until the indicator
|
||
transitions to green (instead of waiting for the 30s poll cycle).
|
||
- Hot/cold model indicators: `/api/chat/ps` proxies Ollama `/api/ps`; ModelSelector shows green/red emoji circles
|
||
- Ollama configured with `OLLAMA_MAX_LOADED_MODELS=2` and `OLLAMA_KEEP_ALIVE=30m`
|
||
- Timeout tuning: connect timeout 30s (cold model loading), warm timeout 300s, pull timeout 1800s
|
||
- SSE reconnection failure shows error toast instead of silently recovering
|
||
- **LLM tool calling:** Models with tool support can create tasks, create/update notes, search and list
|
||
notes/tasks, and manage CalDAV events/todos on behalf of the user during chat. Multi-round tool loop
|
||
(max 5 rounds) allows the LLM to execute tools and then produce a natural language response
|
||
incorporating results. Tool call results are persisted in message `tool_calls` JSONB column and
|
||
rendered as compact ToolCallCard components. SSE emits `tool_call` events for real-time rendering.
|
||
System prompt includes today's date for relative date resolution. Graceful degradation: models without
|
||
tool support respond normally.
|
||
Full tool suite:
|
||
- `create_task` / `create_note` — create new items
|
||
- `update_note` — edit note content (replace/append) AND update task fields: `status`, `priority`,
|
||
`due_date`, `tags` (with `tag_mode`: replace/add/remove); finds by exact title first, falls back to fuzzy search
|
||
- `delete_note` / `delete_task` — permanently delete (require user confirmation via confirm UI;
|
||
validates type so delete_note won't delete a task and vice versa; clears note context cache)
|
||
- `get_note` — retrieve full note body by title/keyword (search_notes only returns 200-char preview)
|
||
- `list_notes` — browse notes by recency/keyword/tags with configurable limit; notes only (use list_tasks for tasks)
|
||
- `list_tasks` — filter tasks by `status`, `priority`, `due_before`, `due_after`, `limit`;
|
||
backed by `list_notes(is_task=True)`; enables "overdue tasks", "high priority", "in progress" queries
|
||
- `search_notes` — keyword search across notes and tasks; optional `type: "note"|"task"` filter
|
||
- Full CalDAV suite: `create_event`, `list_events`, `search_events`, `update_event`, `delete_event`,
|
||
`list_calendars`, `create_todo`, `list_todos`, `search_todos`, `update_todo`, `complete_todo`, `delete_todo`
|
||
(`search_todos` keyword-filters the todo list — companion to `list_todos`)
|
||
- **Streaming status transparency:** The backend emits `status` SSE events at each pipeline stage
|
||
so the user always sees what's happening instead of a blank progress dot. Stages:
|
||
(1) intent ack text streamed as a `chunk` event (tool responses — TTFT ~400ms) or
|
||
`"Generating response..."` status event (chat-only responses);
|
||
(2) human-readable tool label (e.g. `"Creating calendar event..."`) before each tool executes;
|
||
(3) `"Composing response..."` before tool follow-up rounds.
|
||
Frontend: `chat.ts` stores `streamingStatus` ref, cleared on first content chunk or on done/error.
|
||
`ChatView.vue` shows a pulsing dot + italic label above the content while status is active, then
|
||
falls back to the blinking cursor when content streams in. `HomeView.vue` dashboard panel shows
|
||
the status label in place of `...` before any content arrives.
|
||
- **Intent routing (intent-first pipeline, Phase 21):** On the first round, `build_context()` and
|
||
`classify_intent()` run concurrently. Once intent returns (~400ms), the pipeline immediately acts:
|
||
if a tool is detected, the intent's one-sentence `ack` field is streamed as the first chunk
|
||
(becoming TTFT), the tool executes, then the main model generates a follow-up response with the
|
||
tool result. For chat-only responses, the model streams directly with no ack prefix.
|
||
No optimistic streaming queue or race — eliminates wasted GPU prefill when intent won the race.
|
||
`IntentResult.ack` (one-sentence acknowledgment) is embedded in the intent JSON output, so no
|
||
additional LLM call is needed for acknowledgment. Intent model `max_tokens` 350 (was 200).
|
||
Dedicated intent model configurable via `OLLAMA_INTENT_MODEL` env var (default `qwen2.5:7b`)
|
||
or per-user `intent_model` setting — smaller/faster model for routing. Main model default
|
||
is `qwen3:latest` (configurable via `OLLAMA_MODEL` env var or per-user `default_model` setting
|
||
— both settable as dropdowns in the Settings UI populated from installed models).
|
||
`OLLAMA_NUM_CTX` env var (default 8192) controls the KV cache context window for all generation
|
||
calls; reducing from 32768 cuts VRAM usage ~4x without affecting most conversations.
|
||
Supports confidence levels ("high"/"medium"/"low") — low-confidence intents fall through to streaming.
|
||
Passes last 6 user/assistant turns as history for anaphora resolution ("move it", "cancel that").
|
||
Intent router rules cover: update/delete events, CalDAV todos, time-period → list_events (not
|
||
search_events), update_note vs create_note disambiguation, reminder_minutes conversion,
|
||
delete_note vs delete_task disambiguation, get_note for "read/show me this note", list_notes for
|
||
"browse/list notes", tag management via update_note (tag_mode add/remove), search_todos.
|
||
`generate_completion` (used by intent classifier) retries on HTTP 500 (3 attempts, 3s/6s delays)
|
||
to handle cold model loading without failing intent classification.
|
||
- **Intent performance optimizations:**
|
||
- **Intent skip heuristic:** `_should_skip_intent(msg)` in `generation_task.py` skips the intent
|
||
model call entirely for short messages (≤10 words) that contain none of the `_INTENT_TRIGGER_WORDS`
|
||
frozenset (~50 action/object/date words). Saves 400–800ms for conversational replies like "thanks",
|
||
"can you explain that?", "okay" without risking missed tool calls on longer or action-verb messages.
|
||
- **Intent `num_ctx=4096`:** Intent classification calls use a 4k context window (override) instead
|
||
of the default, reducing VRAM pressure and prefill time on every request.
|
||
- **Prior-work fast-path:** `_PRIOR_WORK_REFS` regex in `intent.py` detects phrases like "research
|
||
you did", "note you made", "using your research", "based on the research" etc. and returns no-tool
|
||
immediately — skips the LLM call entirely so the main model answers using `search_notes`/context
|
||
instead of firing a web search. Prevents `search_web` being triggered when the user references
|
||
existing notes.
|
||
- **`search_web` scoping rules:** Intent prompt explicitly prohibits `search_web` for creative/
|
||
brainstorming requests, game design, writing help, or when the user references existing notes.
|
||
Creative/ideation requests ("think of", "come up with", "brainstorm") always route to null (chat).
|
||
`search_web` is only for genuinely new real-time facts not in the user's notes.
|
||
- **Quick-capture endpoint:** `POST /api/quick-capture` in `routes/quick_capture.py` (session auth via
|
||
cookie). Takes `{"text": "natural language string"}`, runs `classify_intent` filtered to creation-only
|
||
tools (`create_note`, `create_task`, `create_event`, `create_todo`), calls `execute_tool`, and returns
|
||
a single synchronous JSON response — no SSE, no conversation ID. Falls back to `create_note` if intent
|
||
is unclear or low-confidence. Response: `{"success", "type", "message", "data", "fallback?"}`.
|
||
Designed for Android/mobile quick-capture use case. Auth: session cookie from `POST /api/auth/login`.
|
||
- **Bug fixes (this session):**
|
||
- `create_note`/`create_task` list-title crash: `execute_tool` now validates `title` is a string and
|
||
returns a structured error so the model self-corrects with individual calls (vs. crashing asyncpg).
|
||
- AI Assist panel 400 errors: `routes/notes.py` assist route was missing `or Config.OLLAMA_MODEL`
|
||
safety net — empty `default_model` DB value was passed to Ollama as-is, causing 400.
|
||
- Note picker → persistent context: paperclip attachment now calls `includeNote()` directly
|
||
(persistent `includedNotes`) instead of the one-shot `attachedNote` pattern.
|
||
- **Default model setting fix:** `delete_setting()` added to `services/settings.py` — saving an empty
|
||
model value in Settings now deletes the DB row rather than storing `""`, so `get_setting()` falls
|
||
back to the `Config` default. Safety net `or Config.OLLAMA_MODEL` guards added in `routes/chat.py`
|
||
status, message, and summarize routes to prevent empty-string model from breaking readiness indicator.
|
||
- **Web research pipeline (Phase 22):** `research.py` implements a full autonomous research pipeline
|
||
triggered by "research X and make a note" (intent routes to `research_topic` tool) or via the 🔍
|
||
Research button in ChatView (sends "Research: {topic}" message).
|
||
Pipeline stages:
|
||
1. Intent model generates 5 focused sub-queries as a JSON array (falls back to `[topic]` on parse failure)
|
||
2. All 5 SearXNG queries execute in parallel with 200ms stagger (avoids hammering rate limiter)
|
||
3. All unique URLs (up to 15) fetched in parallel via `asyncio.gather`; duplicates deduplicated by URL
|
||
4. Failed fetches filtered; up to 12 sources passed to synthesis LLM
|
||
5. Synthesis uses `stream_chat` with `num_ctx=16384`, `num_predict=8192` — tokens stream into
|
||
the chat buffer in real time (user sees the note being written); minimum 2500 words, 6+ topic-appropriate
|
||
sections (topic determines section structure, not hardcoded), detailed prose, `## Sources` section
|
||
6. Note created with `tags=["research"]`; "Research complete!" separator appended to chat
|
||
Also includes `search_web` tool for lightweight single-query searches (no note created; results
|
||
returned to LLM for conversational answer). Both tools only added to tool list when `SEARXNG_URL` is set.
|
||
SearXNG `429` handling: 3-attempt retry with exponential backoff per query.
|
||
Sub-query JSON parsed with `json.JSONDecoder().raw_decode()` to handle trailing model text.
|
||
Frontend: `ToolCallCard.vue` handles `web_search` (external links), `research_pending`, `research_note` types.
|
||
Settings: `GET /api/settings/search?q=` proxy + Search Test section in SettingsView.
|
||
Settings UI: 2-column grid layout (paired simple cards, `full-width` for complex sections).
|
||
Docker: `OLLAMA_NUM_PARALLEL=2` to prevent research and live chat queuing each other.
|
||
SearXNG setup: add app server IP to `botdetection.ip_lists.pass_ip` in SearXNG `settings.yml`
|
||
to bypass rate limiter for trusted backend requests while keeping it active for public users.
|
||
- **CalDAV calendar integration:** Per-user CalDAV settings (URL, username, password, calendar name, timezone).
|
||
LLM tools: `create_event` (all_day, recurrence, timezone, reminder_minutes, attendees, calendar_name),
|
||
`list_events`, `search_events`, `update_event`, `delete_event`, `list_calendars`,
|
||
`create_todo`, `list_todos`, `search_todos`, `update_todo`, `complete_todo`, `delete_todo`.
|
||
All-day events use iCalendar DATE values; recurrence uses RRULE (e.g. `FREQ=YEARLY`).
|
||
VALARM components added for reminders. Attendees via mailto: vCalAddress.
|
||
Multi-calendar search: when no specific calendar configured, all calendars are scanned.
|
||
Runs synchronous caldav library calls in asyncio executor. Settings UI for CalDAV config including timezone.
|
||
- **Dedicated tag chip input (Phase 20):** Tags are now a first-class UI field. `TagInput.vue` is a
|
||
chip-based input placed between the title and body in both editor views. Chips confirmed with
|
||
Enter/comma/autocomplete click; Backspace removes last chip; × removes individual chips. Autocomplete
|
||
fetches from `/api/notes/tags?q=`. Tags saved as explicit `tags: string[]` in create/update payloads.
|
||
`TagSuggestion` Tiptap extension removed; `TagDecoration` kept for legacy inline `#tag` display.
|
||
- **LLM-suggested tags:** Backend service (`tag_suggestions.py`) prompts LLM with existing user tags
|
||
and note content, returns 3-5 relevant tag suggestions filtered against `current_tags` (not body text).
|
||
Exposed via `POST /api/notes/suggest-tags` (accepts `current_tags` list) and `POST /api/notes/:id/append-tag`.
|
||
Integrated in:
|
||
(1) Editor views — "Suggest tags" button shows clickable pills that add chips to TagInput (not body);
|
||
(2) Chat tool calls — `create_note`/`create_task` results include `suggested_tags`, rendered as
|
||
pills in ToolCallCard with one-click apply via append-tag API.
|
||
LLM instructed to use the `tags` parameter, not embed `#tag` text in note body.
|
||
|
||
### Authentication & User Management
|
||
- Session cookie auth with bcrypt, first-user-is-admin, orphaned data claiming
|
||
- Per-user data isolation across all resources
|
||
- Registration auto-closes after first user; admin toggle; invitation-based registration
|
||
- Email invitation system: admin sends invite → branded email → `/register-invite` token flow
|
||
- Password reset: email-based, SHA256-hashed tokens, 1-hour expiry
|
||
- Session cookie hardening: HttpOnly, SameSite=Lax, optional Secure flag
|
||
- Admin user management: list, delete, invite, revoke
|
||
- **OAuth/OIDC SSO (Phase 18):** Authorization Code + PKCE flow via any OIDC provider (Authentik, Keycloak, etc.)
|
||
- `GET /api/auth/oauth/login` → generates state + PKCE verifier, stores in session, redirects to provider
|
||
- `GET /api/auth/oauth/callback` → exchanges code, fetches userinfo, sets session, redirects to `/`
|
||
- Account linking: sub lookup → email auto-link → auto-provision with collision-safe username
|
||
- `LOCAL_AUTH_ENABLED=false` hides password form and blocks `POST /api/auth/login|register`
|
||
- OAuth-only users have `password_hash = NULL`; `authenticate()` returns None for them
|
||
- OIDC discovery response cached in-process after first fetch; no new Python dependencies (`httpx` already present)
|
||
- Config: `OIDC_ISSUER`, `OIDC_CLIENT_ID`, `OIDC_CLIENT_SECRET` (or `_FILE`), `OIDC_SCOPES`
|
||
- **Email change:** `PUT /api/auth/email` — requires current password for local-auth users; OAuth-only users can change freely; checks email uniqueness; updates `authStore.user` in-place
|
||
|
||
### Notifications & Email
|
||
- SMTP email service (aiosmtplib): STARTTLS (587) and implicit TLS (465)
|
||
- Security alerts: login, failed login, logout, password change (fire-and-forget)
|
||
- Task due date reminders: hourly background check, grouped per user, dedup via logs
|
||
- Invitation and password reset emails with branded HTML templates
|
||
- Per-user notification preferences (task reminders, security alerts)
|
||
- Admin SMTP configuration via Settings UI with test email
|
||
- **Shared email template** (`_email_html` in `email.py`): gray outer background, white card, indigo header with inline SVG logo (`_EMAIL_LOGO_SVG`) + "Fabled Assistant" wordmark, content area, footer; used by all six email types (security alert, password reset, reset success, invitation, task reminder, test email)
|
||
|
||
### Logging & Observability
|
||
- Single `app_logs` table: audit (security events), usage (API requests), error (unhandled exceptions)
|
||
- Request logging middleware with timing (skips log endpoints to avoid recursion)
|
||
- Admin log viewer: stats summary, category/search/date filters, paginated table, expandable JSON details
|
||
- Configurable retention via `LOG_RETENTION_DAYS` (default 90), hourly cleanup
|
||
|
||
### Settings & Admin
|
||
- Per-user key-value settings store (assistant name, default model, notification prefs)
|
||
- Admin: backup/restore (full or per-user JSON), SMTP config, base URL, registration toggle, user management, log viewer
|
||
- Configurable base URL for email links (admin setting, env var fallback)
|
||
|
||
### UI & Theming
|
||
- Dark/light theme with CSS custom properties, design tokens, `prefers-color-scheme` detection; light mode uses indigo brand color (`--color-primary: #6366f1`) and indigo-tinted off-white backgrounds (`--color-bg: #f5f5fb`, `--color-bg-secondary: #ededf5`) with white cards to pop content
|
||
- Responsive design: breakpoints (480/768/1024), hamburger menu, mobile touch targets (44px), sidebar overlays
|
||
- App shell: navbar always visible, 100dvh flex layout, all views fit within viewport
|
||
- Toast notifications (success/error/warning, 4s auto-dismiss)
|
||
- DOMPurify sanitization on all rendered markdown
|
||
- **Site-wide max-width:** List/chat/settings views 1200px; editor pages 1400px; note/task viewer layout 1400px (content area 1100px)
|
||
- **Favicon:** `frontend/public/favicon.svg` — light mode uses indigo brand color (`#6366f1` fill, `#4f46e5` stroke, `#a5b4fc` lines); dark mode uses light gray; gold sparkle always visible; `@media (prefers-color-scheme: dark)` in SVG `<style>`
|
||
|
||
### Infrastructure
|
||
- Multi-stage Dockerfile: Node build → Python runtime, auto-migration on startup
|
||
- Docker Compose (dev) with app service healthcheck (`/api/health`, 10s interval, 30s start period) + Docker Swarm production stack with secrets, overlay networks, health checks
|
||
- `.dockerignore` prevents secrets, `node_modules`, `__pycache__`, `.env.*` from leaking into build context
|
||
- Production compose (`docker-compose.prod.yml`): `TRUST_PROXY_HEADERS=true` + `SECURE_COOKIES=true` set for Traefik deployments; port 5000 not published directly (Traefik routes internally)
|
||
- Quart serves Vue SPA from static + REST API under `/api/`; 404 handler for SPA routing
|
||
- Security headers applied in `after_request`: `X-Content-Type-Options`, `X-Frame-Options: DENY`, `Referrer-Policy`, `Content-Security-Policy`
|
||
- In-memory sliding-window rate limiter on all auth endpoints (login, register, forgot/reset password); proxy-aware client IP with `TRUST_PROXY_HEADERS`
|
||
- SQLAlchemy async engine configured with `pool_pre_ping=True` (tests connections before use, discards stale ones after Postgres restart) and `pool_recycle=1800` (recycles idle connections every 30 min to prevent TCP/firewall staleness)
|
||
- Alembic migrations: 15 migrations, all raw SQL with idempotency guards (IF NOT EXISTS, DO $$ BEGIN...EXCEPTION)
|
||
- Config from env vars + Docker secrets file support (`_read_secret`)
|
||
|
||
## Development Workflow
|
||
- All development and testing done via Docker: `docker compose up --build`
|
||
- No local dependency installation — everything containerized
|
||
- Frontend dev: Vite dev server with proxy to Quart (via Docker)
|
||
- Production build: Dockerfile multi-stage — Vite builds Vue into static files,
|
||
Quart serves them
|
||
- To reset database: `docker compose down -v && docker compose up --build`
|
||
|
||
## Backlog
|
||
- Calendar/timeline view for tasks
|
||
- Import/export (Markdown files, JSON)
|
||
- Email integration (read/send emails from chat via IMAP/SMTP tools)
|
||
- Session invalidation on user deletion
|
||
- **Note relation map (Obsidian-style graph):** Interactive force-directed graph visualizing connections between
|
||
notes via wikilinks (`[[Title]]`) and shared tags. Nodes = notes/tasks; edges = wikilink references or
|
||
tag co-occurrence. Clicking a node navigates to that note. Filter by tag or depth. Useful for discovering
|
||
clusters of related notes and orphaned notes with no connections.
|