FabledScribe/docs/architecture.md

Architecture

Stack

Layer	Technology	Notes
Frontend	Vue 3 + TypeScript + Vite + Pinia + Vue Router	SPA served from the same container as the API
Editor	Tiptap (ProseMirror) with custom slash-command extension
Backend	Python 3.12, Quart (async ASGI)	Serves both API and built frontend static files
Database	PostgreSQL 16, SQLAlchemy 2.0 async, Alembic	asyncpg driver
LLM	Ollama (local)	Any OpenAI-compatible API also works
Search	SearXNG (optional, self-hosted)	Web search + image search
Push	Web Push / VAPID (pywebpush 2.x)
Deployment	Docker Compose	Single-container app + separate DB + LLM service

High-Level Component Diagram

┌─────────────────────────────────────────────┐
│              Docker Compose                  │
│                                              │
│  ┌──────────────────────┐   ┌────────────┐  │
│  │  fabledassistant     │   │  ollama    │  │
│  │  ┌────────────────┐  │   │            │  │
│  │  │  Quart Server  │  │   │  LLM API   │  │
│  │  │  ┌──────────┐  │  │   │            │  │
│  │  │  │ Vue SPA  │  │  │   └────────────┘  │
│  │  │  │ (static) │  │  │         ▲         │
│  │  │  └──────────┘  │  │         │         │
│  │  │  ┌──────────┐  │  │    HTTP/REST      │
│  │  │  │ /api/*   │──┼──┼─────────┘         │
│  │  │  └──────────┘  │  │                   │
│  │  │       │        │  │  ┌────────────┐   │
│  │  │       ▼        │  │  │ PostgreSQL │   │
│  │  │  ┌──────────┐  │  │  │    16      │   │
│  │  │  │ asyncpg  │──┼──┼──▶            │   │
│  │  │  └──────────┘  │  │  └────────────┘   │
│  │  └────────────────┘  │                   │
│  └──────────────────────┘                   │
└─────────────────────────────────────────────┘

Project Structure

fabledassistant/
├── docker-compose.yml           # Development stack
├── docker-compose.prod.yml      # Production stack (Docker Swarm)
├── Dockerfile                   # Multi-stage build (Node → Python)
├── alembic/                     # Database migrations
│   └── versions/                # Migration files (idempotent raw SQL)
├── fable-mcp/                   # Fable MCP server package
│   └── fable_mcp/
│       ├── server.py            # FastMCP tool registrations
│       ├── client.py            # FableClient (httpx wrapper)
│       └── tools/               # Tool modules (notes, tasks, projects, …)
├── src/fabledassistant/
│   ├── app.py                   # Quart app factory + blueprint registration
│   ├── config.py                # Config class (reads env vars)
│   ├── auth.py                  # login_required decorator, session checks
│   ├── models/                  # SQLAlchemy models
│   ├── routes/                  # API blueprints (one file per resource)
│   ├── services/                # Business logic (access, llm, tools, sharing, …)
│   └── static/                  # Built Vue SPA (generated at Docker build time)
└── frontend/
    └── src/
        ├── views/               # Page-level Vue components
        ├── components/          # Reusable UI components
        ├── composables/         # Vue composables (autosave, shortcuts, …)
        ├── stores/              # Pinia stores (auth, chat, notes, notifications, …)
        └── api/                 # Typed API client (client.ts)

Key Design Decisions

Single container for frontend + API. Quart serves the Vue.js production build as static files and exposes the REST API under /api/. The SPA is built by Vite during the Docker image build.

SPA routing via 404 handler. app.py uses @app.errorhandler(404) (not a catch-all route) to serve static files or fall back to index.html. API routes (/api/*) always return JSON 404. This avoids a catch-all /<path:path> route intercepting API GETs.

Unified note/task model. A task is just a note with task attributes enabled. status IS NOT NULL means it's a task. "Convert to task" sets status='todo'; "convert to note" clears status, priority, due_date. No separate table, no cascade complexity.

First-class tag column. Tags live in a tags ARRAY[text] column and are explicitly set by the client — not auto-extracted from body text. Hierarchical tags (project/webapp) supported via SQL unnest + LIKE prefix matching.

Background generation architecture. LLM streaming runs in a detached asyncio.Task that writes into an in-memory GenerationBuffer. SSE clients tail the buffer and can reconnect mid-stream without data loss. Buffer has a cancel_event for user-initiated stop. Completed buffers are cleaned up after 60s grace period. Periodic DB flushes every 5s preserve partial content. Both chat and AI Assist use this architecture.

SSE over WebSockets for LLM streaming. SSE clients connect via GET /api/chat/conversations/:id/generation/stream with Last-Event-ID reconnection support. Frontend uses fetch() + ReadableStream.

Context building is server-side. Backend fetches URL content and searches notes. Frontend sends the message text + optional context note IDs. build_context() returns (messages, context_meta); metadata includes auto-found note IDs/titles sent to frontend via a context SSE event before streaming begins.

No blocking long-running operations. Any slow operation (model pulls, LLM calls, URL fetching) must never block app startup or freeze the UI. Backend uses SSE streaming for incremental responses. Model pulls stream NDJSON progress to the frontend.

SSRF protection. services/llm.py blocks requests to loopback, private, link-local, reserved, and multicast addresses before fetching. follow_redirects=False prevents redirect-based bypasses.

Session cookie security. HttpOnly and SameSite=Lax always set. Secure flag controlled by SECURE_COOKIES env var.

Rate limiting. In-memory sliding-window rate limiter (rate_limit.py) applied to auth endpoints: login (10/60s), register (5/300s), forgot-password (5/300s), reset-password (10/60s). Keys are per-IP. TRUST_PROXY_HEADERS env var enables X-Forwarded-For / X-Real-IP when behind a reverse proxy.

Idempotent migrations. All Alembic migrations use raw SQL with CREATE TABLE IF NOT EXISTS, CREATE INDEX IF NOT EXISTS, and DO $$ BEGIN CREATE TYPE ... EXCEPTION WHEN duplicate_object to allow safe re-runs.

Data Model

Users

Column	Type	Notes
id	int PK
username	text UNIQUE NOT NULL
email	text nullable
password_hash	text nullable	NULL for OAuth-only accounts
oauth_sub	text UNIQUE nullable	OIDC subject identifier
role	text NOT NULL DEFAULT 'user'	First user auto-assigned 'admin'
session_version	int NOT NULL DEFAULT 1	Bumped on password change to evict sessions
created_at	timestamptz

Notes (unified — includes tasks)

Column	Type	Notes
id	int PK
title	text
body	text	Markdown
tags	ARRAY[text]	GIN indexed; explicitly set by client
parent_id	int FK self nullable	Sub-tasks / sub-notes
user_id	int FK users nullable	CASCADE
project_id	int FK projects nullable
milestone_id	int FK milestones nullable
status	text nullable	todo/in_progress/done — non-null = task
priority	text nullable	none/low/medium/high
due_date	date nullable
created_at, updated_at	timestamptz

Indexes: GIN on tags, B-tree on status, B-tree on title.

Settings

Composite PK (user_id, key). Per-user key-value store. CRUD via services/settings.py. Used for: default_model, assistant_name, briefing_enabled, briefing_locations, office_days, etc.

Conversations / Messages

conversations: id, title, model, user_id, conversation_type (chat/briefing/mcp), briefing_date, created_at, updated_at. messages: id, conversation_id FK CASCADE, role (user/assistant), content, status (done/generating), created_at.

Title auto-generated by LLM on first exchange, re-generated every 10th message.

Projects / Milestones

projects: id, user_id, title, description, goal, status (active/completed/archived), color, timestamps. milestones: id, user_id, project_id FK CASCADE, title, description, status, order_index, timestamps.

Sharing & Access

project_shares, note_shares: each has shared_with_user_id OR shared_with_group_id (exclusive), permission (viewer/editor/admin), invited_by.

groups, group_memberships: platform-wide groups with member/owner roles.

Permission resolution is centralised in services/access.py. get_project_permission(uid, project_id) checks ownership → direct share → group-based share → note→project inheritance, returning the highest applicable permission.

Briefing-Related Tables

rss_feeds: id, user_id, url, name, category, last_fetched_at. rss_items: id, feed_id FK, guid, title, url, summary, pub_date. weather_cache: per-user cache with lat, lon, location_name, forecast_json, fetched_at.

App Logs

category (audit/usage/error), user_id FK nullable (SET NULL on delete), username (denormalised), action, endpoint, method, status_code, duration_ms, error_type, error_message, traceback, details JSONB.

Detailed File Reference

Backend (src/fabledassistant/)

File	Responsibility
app.py	Quart app factory; SPA via 404 handler; JSON 404/500 for API; request logging; security headers in after_request
auth.py	login_required, admin_required, get_current_user_id — shared _check_auth() helper
config.py	All config from env vars + Docker secret file support (_read_secret); SECURE_COOKIES, TRUST_PROXY_HEADERS, OLLAMA_NUM_CTX; oidc_enabled(), searxng_enabled() classmethods
rate_limit.py	In-memory sliding-window rate limiter (asyncio.Lock + defaultdict); is_rate_limited(key, max, window)
models/note.py	Unified Note model (notes + tasks)
models/user.py	id, username, email, password_hash (nullable — NULL for OAuth-only), oauth_sub (unique nullable), role, session_version
models/conversation.py	Conversation + Message models
models/app_log.py	AppLog with category, denormalised username, details JSONB
routes/api.py	/api blueprint; GET /api/health (public)
routes/auth.py	Register, login, logout, me, password/email change, password reset, invite registration, OAuth login+callback; rate limiting; LOCAL_AUTH_ENABLED guards
routes/admin.py	Backup, restore, user management, registration toggle, invitations, base URL, SMTP (admin only)
routes/chat.py	Conversations CRUD; SSE generation stream; model pull/delete/list/warm; briefing conversation routes
routes/notes.py	Notes CRUD + wikilinks + backlinks + assist + link suggestions + version history + drafts
routes/tasks.py	Tasks CRUD; POST accepts project name string (resolved to project_id)
routes/quick_capture.py	POST /api/quick-capture — single-shot natural language item creation
routes/search.py	GET /api/search — semantic + keyword hybrid search
services/auth.py	create_user, authenticate, user lookups, password reset tokens, invitation tokens
services/oauth.py	OIDC discovery (cached), PKCE auth URL, code exchange, find_or_create_oauth_user
services/llm.py	build_context(), RAG injection, history summarisation, stream_chat_with_tools(), URL fetching, SSRF guard
services/generation_task.py	run_generation() — full chat pipeline: intent routing, tool loop, SSE fan-out, push notification; run_assist_generation()
services/intent.py	classify_intent() — fast non-streaming LLM call; intent skip heuristic; _PRIOR_WORK_REFS fast-path
services/tools.py	All LLM tool definitions + execute_tool() dispatcher; duplicate guards; _resolve_project() 4-step lookup
services/embeddings.py	upsert_note_embedding(), semantic_search_notes() (pgvector cosine similarity)
services/generation_buffer.py	In-memory SSE event buffer; cancel_event; 60s cleanup; supports both chat (int keys) and assist (string keys)
services/briefing_pipeline.py	Two-lane parallel gather → LLM synthesis → GenerationBuffer stream
services/briefing_scheduler.py	APScheduler BackgroundScheduler; slots with catch-up logic; async-safe via asyncio.create_task
services/research.py	SearXNG research pipeline: 5 sub-queries → parallel fetch → synthesis; search_images for image category
services/caldav.py	Full CalDAV event/todo lifecycle; synchronous library calls in asyncio executor
services/images.py	fetch_and_store_image() — SHA-256 dedup, content-type validation, 5 MB cap
services/backup.py	export_full_backup(), export_user_backup(), restore_full_backup() (version 2 with ID maps)
services/push.py	VAPID key auto-generation; send_push_notification() fire-and-forget; 410 Gone cleanup
services/logging.py	log_audit, log_usage, log_error, start_log_retention_loop (hourly cleanup)
services/weather.py	Nominatim geocoding + Open-Meteo forecast + per-user DB cache
services/rss.py	feedparser fetch; per-feed DB cache; prune-to-100 items
services/assist.py	build_assist_messages() — section/whole-doc modes; project context injection

Frontend (frontend/src/)

File	Responsibility
App.vue	App shell (100dvh flex column); global keyboard shortcuts (g+key nav, n/t/c/e///?); starts status polling + loads settings on mount
api/client.ts	ApiError, apiGet/Post/Put/Patch/Delete, apiSSEStream (fetch + ReadableStream), auto 401→login redirect
stores/auth.ts	User, isAuthenticated, isAdmin, oauthEnabled, localAuthEnabled, login/register/logout/checkAuth
stores/chat.ts	Conversation CRUD; sendMessage() SSE streaming; reconnectIfGenerating(); message queue (localStorage); streamingStatus
stores/notes.ts	CRUD + tag filter; resolveTitle; convertToTask/Note; fetchBacklinks; fetchAllTags
stores/tasks.ts	CRUD + status/priority filter; patchStatus
stores/settings.ts	assistantName, defaultModel, installedModels; pullModel(), deleteModel()
stores/push.ts	isSupported, permission, isSubscribed, subscribe/unsubscribe
stores/notifications.ts	count, items, fetchCount, fetchAll, markRead, markAll
views/HomeView.vue	Chat-first dashboard; 6 task sections (Overdue → Other); 8 recent notes; inline streaming response
views/ChatView.vue	/chat — SSE streaming; context sidebar (Suggested/In Context); note picker; model selector; message queue; bulk delete
views/WorkspaceView.vue	/workspace/:id — 3-panel (tasks/chat/notes); SSE tool-call watcher; conv persisted to localStorage
views/GraphView.vue	/graph — D3 force-directed; tag/note/project-hub nodes; physics panel; peek panel
views/ProjectView.vue	Kanban grouped by milestone; advance buttons; milestone management
views/SettingsView.vue	Tabbed settings (11 tabs); tab state in localStorage
views/NoteEditorView.vue	Tiptap editor; 2-column layout; AI assist panel; diff view; version history; autosave
views/TaskEditorView.vue	Tiptap editor; 2-column layout; AI assist panel; task log section; sub-tasks
components/ToolCallCard.vue	Tool call results; requires_confirmation inline confirm/deny; direct POST on "Create anyway"
components/ChatMessage.vue	Message bubble; markdown rendering; tool call cards; "Save as Note"
components/TagInput.vue	Chip-based tag input; Enter/comma to confirm; autocomplete from /api/notes/tags
components/TiptapEditor.vue	Tiptap wrapper; markdown↔HTML round-trip; selection change emit; WikilinkDecoration; TagDecoration
components/ShareDialog.vue	<Teleport> modal; user tab + group tab; current shares list
components/WorkspaceTaskPanel.vue	Milestone-grouped task list; detail slide-over
components/WorkspaceNoteEditor.vue	List ↔ TipTap editor with autosave
composables/useAssist.ts	AI assist: section parsing, SSE streaming, accept/reject, LCS diff, persistent draft
composables/useAutoSave.ts	Interval-based autosave (5 min) with dirty/saving guards
composables/useEditorGuards.ts	Ctrl+S, beforeunload warning, route-leave confirm
composables/useTagSuggestions.ts	/api/notes/suggest-tags call + suggestion state
composables/useListKeyboardNavigation.ts	j/k/Enter list navigation; focus-in-input guard
extensions/WikilinkDecoration.ts	ProseMirror decoration plugin highlighting [[wikilinks]]
extensions/TagDecoration.ts	ProseMirror decoration plugin highlighting #tags
extensions/WikilinkSuggestion.ts	@tiptap/suggestion extension for [[ autocomplete

Key Services

Service	Responsibility
services/access.py	Permission resolution for all shared resources
services/llm.py	build_context(), RAG injection, history summarisation
services/generation_task.py	SSE streaming, tool-call loop, GenerationBuffer management
services/tools.py	All LLM tool implementations (create_note, search_notes, get_weather, …)
services/embeddings.py	upsert_note_embedding(), semantic_search_notes()
services/briefing_pipeline.py	Two-lane parallel gather → LLM synthesis → briefing output
services/briefing_scheduler.py	APScheduler integration, catch-up logic for missed slots
services/backup.py	Full and per-user backup export/restore (version 2 format)
services/weather.py	Nominatim geocoding + Open-Meteo forecast fetch + DB cache
services/rss.py	feedparser-based fetch, per-feed DB cache, prune-to-100

Authentication

Local username/password auth and OIDC/OAuth (PKCE). services/oauth.py handles discovery and find_or_create_oauth_user. On OAuth login: checks existing oauth_sub → matching email → creates new user.

Session cookies: HttpOnly, SameSite=Lax, optionally Secure. Session includes session_version; mismatch with DB value (after password change) results in 401 and session clear.

See sso-oauth.md for provider-specific setup instructions.

RAG Pipeline

1. semantic_search_notes() — cosine similarity via pgvector, threshold configurable per call.
2. Notes ≥ 0.60 similarity auto-injected into system prompt (up to 3, 800 chars each).
3. Notes 0.45–0.60 surfaced in chat sidebar as "Suggested" (user clicks to include).
4. Explicitly included notes delivered full-body.
5. Project-scoped RAG: rag_project_id param restricts all searches to a project's notes.
6. excluded_note_ids prevents the current note from being injected as its own context.

Embedding model: nomic-embed-text via Ollama. Backfill runs 30s after startup (background task).