FabledScribe/docs/architecture.md

Architecture

Stack

Layer	Technology	Notes
Frontend	Vue 3 + TypeScript + Vite + Pinia + Vue Router	SPA served from the same container as the API
Editor	Tiptap (ProseMirror) with custom slash-command extension
Backend	Python 3.12, Quart (async ASGI)	Serves both API and built frontend static files
Database	PostgreSQL 16, SQLAlchemy 2.0 async, Alembic	asyncpg driver
LLM	Ollama (local)	Any OpenAI-compatible API also works
Search	SearXNG (optional, self-hosted)	Web search + image search
Push	Web Push / VAPID (pywebpush 2.x)
Deployment	Docker Compose	Single-container app + separate DB + LLM service

High-Level Component Diagram

┌─────────────────────────────────────────────┐
│              Docker Compose                  │
│                                              │
│  ┌──────────────────────┐   ┌────────────┐  │
│  │  fabledassistant     │   │  ollama    │  │
│  │  ┌────────────────┐  │   │            │  │
│  │  │  Quart Server  │  │   │  LLM API   │  │
│  │  │  ┌──────────┐  │  │   │            │  │
│  │  │  │ Vue SPA  │  │  │   └────────────┘  │
│  │  │  │ (static) │  │  │         ▲         │
│  │  │  └──────────┘  │  │         │         │
│  │  │  ┌──────────┐  │  │    HTTP/REST      │
│  │  │  │ /api/*   │──┼──┼─────────┘         │
│  │  │  └──────────┘  │  │                   │
│  │  │       │        │  │  ┌────────────┐   │
│  │  │       ▼        │  │  │ PostgreSQL │   │
│  │  │  ┌──────────┐  │  │  │    16      │   │
│  │  │  │ asyncpg  │──┼──┼──▶            │   │
│  │  │  └──────────┘  │  │  └────────────┘   │
│  │  └────────────────┘  │                   │
│  └──────────────────────┘                   │
└─────────────────────────────────────────────┘

Project Structure

fabledassistant/
├── docker-compose.yml           # Development stack
├── docker-compose.prod.yml      # Production stack (Docker Swarm)
├── Dockerfile                   # Multi-stage build (Node → Python)
├── alembic/                     # Database migrations
│   └── versions/                # Migration files (idempotent raw SQL)
├── fable-mcp/                   # Fable MCP server package
│   └── fable_mcp/
│       ├── server.py            # FastMCP tool registrations
│       ├── client.py            # FableClient (httpx wrapper)
│       └── tools/               # Tool modules (notes, tasks, projects, …)
├── src/fabledassistant/
│   ├── app.py                   # Quart app factory + blueprint registration
│   ├── config.py                # Config class (reads env vars)
│   ├── auth.py                  # login_required decorator, session checks
│   ├── models/                  # SQLAlchemy models
│   ├── routes/                  # API blueprints (one file per resource)
│   ├── services/                # Business logic (access, llm, tools, sharing, …)
│   └── static/                  # Built Vue SPA (generated at Docker build time)
└── frontend/
    └── src/
        ├── views/               # Page-level Vue components
        ├── components/          # Reusable UI components
        ├── composables/         # Vue composables (autosave, shortcuts, …)
        ├── stores/              # Pinia stores (auth, chat, notes, notifications, …)
        └── api/                 # Typed API client (client.ts)

Key Design Decisions

Single container for frontend + API. Quart serves the Vue.js production build as static files and exposes the REST API under /api/. The SPA is built by Vite during the Docker image build.

SPA routing via 404 handler. app.py uses @app.errorhandler(404) (not a catch-all route) to serve static files or fall back to index.html. API routes (/api/*) always return JSON 404. This avoids a catch-all /<path:path> route intercepting API GETs.

Unified note/task model. A task is just a note with task attributes enabled. status IS NOT NULL means it's a task. "Convert to task" sets status='todo'; "convert to note" clears status, priority, due_date. No separate table, no cascade complexity.

First-class tag column. Tags live in a tags ARRAY[text] column and are explicitly set by the client — not auto-extracted from body text. Hierarchical tags (project/webapp) supported via SQL unnest + LIKE prefix matching.

Background generation architecture. LLM streaming runs in a detached asyncio.Task that writes into an in-memory GenerationBuffer. SSE clients tail the buffer and can reconnect mid-stream without data loss. Buffer has a cancel_event for user-initiated stop. Completed buffers are cleaned up after 60s grace period. Periodic DB flushes every 5s preserve partial content. Both chat and AI Assist use this architecture.

SSE over WebSockets for LLM streaming. SSE clients connect via GET /api/chat/conversations/:id/generation/stream with Last-Event-ID reconnection support. Frontend uses fetch() + ReadableStream.

Context building is server-side. Backend fetches URL content and searches notes. Frontend sends the message text + optional context note IDs. build_context() returns (messages, context_meta); metadata includes auto-found note IDs/titles sent to frontend via a context SSE event before streaming begins.

No blocking long-running operations. Any slow operation (model pulls, LLM calls, URL fetching) must never block app startup or freeze the UI. Backend uses SSE streaming for incremental responses. Model pulls stream NDJSON progress to the frontend.

SSRF protection. services/llm.py blocks requests to loopback, private, link-local, reserved, and multicast addresses before fetching. follow_redirects=False prevents redirect-based bypasses.

Session cookie security. HttpOnly and SameSite=Lax always set. Secure flag controlled by SECURE_COOKIES env var.

Rate limiting. In-memory sliding-window rate limiter (rate_limit.py) applied to auth endpoints: login (10/60s), register (5/300s), forgot-password (5/300s), reset-password (10/60s). Keys are per-IP. TRUST_PROXY_HEADERS env var enables X-Forwarded-For / X-Real-IP when behind a reverse proxy.

Idempotent migrations. All Alembic migrations use raw SQL with CREATE TABLE IF NOT EXISTS, CREATE INDEX IF NOT EXISTS, and DO $$ BEGIN CREATE TYPE ... EXCEPTION WHEN duplicate_object to allow safe re-runs.

Data Model

Users

Column	Type	Notes
id	int PK
username	text UNIQUE NOT NULL
email	text nullable
password_hash	text nullable	NULL for OAuth-only accounts
oauth_sub	text UNIQUE nullable	OIDC subject identifier
role	text NOT NULL DEFAULT 'user'	First user auto-assigned 'admin'
session_version	int NOT NULL DEFAULT 1	Bumped on password change to evict sessions
created_at	timestamptz

Notes (unified — includes tasks)

Column	Type	Notes
id	int PK
title	text
body	text	Markdown
tags	ARRAY[text]	GIN indexed; explicitly set by client
parent_id	int FK self nullable	Sub-tasks / sub-notes
user_id	int FK users nullable	CASCADE
project_id	int FK projects nullable
milestone_id	int FK milestones nullable
status	text nullable	todo/in_progress/done — non-null = task
priority	text nullable	none/low/medium/high
due_date	date nullable
created_at, updated_at	timestamptz

Indexes: GIN on tags, B-tree on status, B-tree on title.

Settings

Composite PK (user_id, key). Per-user key-value store. CRUD via services/settings.py. Used for: default_model, assistant_name, briefing_enabled, briefing_locations, office_days, etc.

Conversations / Messages

conversations: id, title, model, user_id, conversation_type (chat/briefing/mcp), briefing_date, created_at, updated_at. messages: id, conversation_id FK CASCADE, role (user/assistant), content, status (done/generating), created_at.

Title auto-generated by LLM on first exchange, re-generated every 10th message.

Projects / Milestones

projects: id, user_id, title, description, goal, status (active/completed/archived), color, timestamps. milestones: id, user_id, project_id FK CASCADE, title, description, status, order_index, timestamps.

Sharing & Access

project_shares, note_shares: each has shared_with_user_id OR shared_with_group_id (exclusive), permission (viewer/editor/admin), invited_by.

groups, group_memberships: platform-wide groups with member/owner roles.

Permission resolution is centralised in services/access.py. get_project_permission(uid, project_id) checks ownership → direct share → group-based share → note→project inheritance, returning the highest applicable permission.

Briefing-Related Tables

rss_feeds: id, user_id, url, name, category, last_fetched_at. rss_items: id, feed_id FK, guid, title, url, summary, pub_date. weather_cache: per-user cache with lat, lon, location_name, forecast_json, fetched_at.

App Logs

category (audit/usage/error), user_id FK nullable (SET NULL on delete), username (denormalised), action, endpoint, method, status_code, duration_ms, error_type, error_message, traceback, details JSONB.

Key Services

Service	Responsibility
services/access.py	Permission resolution for all shared resources
services/llm.py	build_context(), RAG injection, history summarisation
services/generation_task.py	SSE streaming, tool-call loop, GenerationBuffer management
services/tools.py	All LLM tool implementations (create_note, search_notes, get_weather, …)
services/embeddings.py	upsert_note_embedding(), semantic_search_notes()
services/briefing_pipeline.py	Two-lane parallel gather → LLM synthesis → briefing output
services/briefing_scheduler.py	APScheduler integration, catch-up logic for missed slots
services/backup.py	Full and per-user backup export/restore (version 2 format)
services/weather.py	Nominatim geocoding + Open-Meteo forecast fetch + DB cache
services/rss.py	feedparser-based fetch, per-feed DB cache, prune-to-100

Authentication

Local username/password auth and OIDC/OAuth (PKCE). services/oauth.py handles discovery and find_or_create_oauth_user. On OAuth login: checks existing oauth_sub → matching email → creates new user.

Session cookies: HttpOnly, SameSite=Lax, optionally Secure. Session includes session_version; mismatch with DB value (after password change) results in 401 and session clear.

See sso-oauth.md for provider-specific setup instructions.

RAG Pipeline

1. semantic_search_notes() — cosine similarity via pgvector, threshold configurable per call.
2. Notes ≥ 0.60 similarity auto-injected into system prompt (up to 3, 800 chars each).
3. Notes 0.45–0.60 surfaced in chat sidebar as "Suggested" (user clicks to include).
4. Explicitly included notes delivered full-body.
5. Project-scoped RAG: rag_project_id param restricts all searches to a project's notes.
6. excluded_note_ids prevents the current note from being injected as its own context.

Embedding model: nomic-embed-text via Ollama. Backfill runs 30s after startup (background task).