diff --git a/docs/architecture.md b/docs/architecture.md index 4a37bf3..d6ee9fd 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -161,6 +161,10 @@ Permission resolution is centralised in `services/access.py`. `get_project_permi `rss_items`: `id`, `feed_id` FK, `guid`, `title`, `url`, `summary`, `pub_date`. `weather_cache`: per-user cache with `lat`, `lon`, `location_name`, `forecast_json`, `fetched_at`. +### API Keys + +`api_keys`: `id`, `user_id` FK CASCADE, `prefix` (first 8 chars, displayed in UI), `key_hash` (SHA-256 of full key — full key never stored), `name`, `scope` (`read`/`write`), `created_at`, `last_used_at`. + ### App Logs `category` (`audit`/`usage`/`error`), `user_id` FK nullable (SET NULL on delete), `username` (denormalised), `action`, `endpoint`, `method`, `status_code`, `duration_ms`, `error_type`, `error_message`, `traceback`, `details` JSONB. @@ -172,37 +176,68 @@ Permission resolution is centralised in `services/access.py`. `get_project_permi | File | Responsibility | |------|---------------| | `app.py` | Quart app factory; SPA via 404 handler; JSON 404/500 for API; request logging; security headers in `after_request` | -| `auth.py` | `login_required`, `admin_required`, `get_current_user_id` — shared `_check_auth()` helper | +| `auth.py` | `login_required`, `admin_required`, `get_current_user_id` — shared `_check_auth()` helper; accepts session cookie or `Authorization: Bearer ` | | `config.py` | All config from env vars + Docker secret file support (`_read_secret`); `SECURE_COOKIES`, `TRUST_PROXY_HEADERS`, `OLLAMA_NUM_CTX`; `oidc_enabled()`, `searxng_enabled()` classmethods | | `rate_limit.py` | In-memory sliding-window rate limiter (`asyncio.Lock` + `defaultdict`); `is_rate_limited(key, max, window)` | | `models/note.py` | Unified Note model (notes + tasks) | | `models/user.py` | `id`, `username`, `email`, `password_hash` (nullable — NULL for OAuth-only), `oauth_sub` (unique nullable), `role`, `session_version` | | `models/conversation.py` | `Conversation` + `Message` models | | `models/app_log.py` | `AppLog` with `category`, denormalised `username`, `details` JSONB | +| `models/api_key.py` | `ApiKey`: `id`, `user_id`, `prefix`, `key_hash` (SHA-256), `name`, `scope` (`read`/`write`), `created_at`, `last_used_at` | +| `models/event.py` | Dead code — CalDAV/Radicale internal events table. Radicale was trialled and removed; `calendar_sync.py` is also dead. | | `routes/api.py` | `/api` blueprint; `GET /api/health` (public) | | `routes/auth.py` | Register, login, logout, me, password/email change, password reset, invite registration, OAuth login+callback; rate limiting; `LOCAL_AUTH_ENABLED` guards | | `routes/admin.py` | Backup, restore, user management, registration toggle, invitations, base URL, SMTP (admin only) | | `routes/chat.py` | Conversations CRUD; SSE generation stream; model pull/delete/list/warm; briefing conversation routes | | `routes/notes.py` | Notes CRUD + wikilinks + backlinks + assist + link suggestions + version history + drafts | | `routes/tasks.py` | Tasks CRUD; `POST` accepts `project` name string (resolved to `project_id`) | +| `routes/task_logs.py` | Task work log CRUD (`GET/POST/DELETE /api/tasks/:id/logs`) | +| `routes/projects.py` | Projects CRUD + summary endpoint | +| `routes/milestones.py` | Milestones CRUD under `/api/projects/:id/milestones` | +| `routes/settings.py` | Per-user settings key-value (`GET/PUT /api/settings/:key`) | +| `routes/briefing.py` | Briefing conversation + reply + history; RSS feed management | +| `routes/groups.py` | Group CRUD + membership management (admin) | +| `routes/shares.py` | Share project/note with user or group; revoke; list incoming shares | +| `routes/in_app_notifications.py` | In-app notification list + mark-read + count | +| `routes/push.py` | Web Push subscription subscribe/unsubscribe; VAPID public key | +| `routes/users.py` | User profile; admin user list + delete | +| `routes/images.py` | Serve cached images at `/api/images/` | +| `routes/export.py` | `GET /api/export` — personal Markdown ZIP or JSON array download | +| `routes/api_keys.py` | API key CRUD (`GET/POST/DELETE /api/api-keys`) | +| `routes/fable_mcp_dist.py` | `GET /api/fable-mcp/info` + `GET /api/fable-mcp/download` — package distribution | | `routes/quick_capture.py` | `POST /api/quick-capture` — single-shot natural language item creation | | `routes/search.py` | `GET /api/search` — semantic + keyword hybrid search | | `services/auth.py` | `create_user`, `authenticate`, user lookups, password reset tokens, invitation tokens | | `services/oauth.py` | OIDC discovery (cached), PKCE auth URL, code exchange, `find_or_create_oauth_user` | +| `services/api_keys.py` | `generate_key()`, `create_api_key()`, `list_api_keys()`, `revoke_api_key()`, `lookup_key()` (SHA-256 hash lookup) | | `services/llm.py` | `build_context()`, RAG injection, history summarisation, `stream_chat_with_tools()`, URL fetching, SSRF guard | | `services/generation_task.py` | `run_generation()` — full chat pipeline: intent routing, tool loop, SSE fan-out, push notification; `run_assist_generation()` | | `services/intent.py` | `classify_intent()` — fast non-streaming LLM call; intent skip heuristic; `_PRIOR_WORK_REFS` fast-path | | `services/tools.py` | All LLM tool definitions + `execute_tool()` dispatcher; duplicate guards; `_resolve_project()` 4-step lookup | | `services/embeddings.py` | `upsert_note_embedding()`, `semantic_search_notes()` (pgvector cosine similarity) | | `services/generation_buffer.py` | In-memory SSE event buffer; `cancel_event`; 60s cleanup; supports both chat (int keys) and assist (string keys) | +| `services/notes.py` | Note CRUD, wikilink resolution, backlink queries, tag management | +| `services/note_versions.py` | Version snapshot on save; restore-from-version; diff metadata | +| `services/note_drafts.py` | Per-user per-note draft persistence (AI Assist pending state) | +| `services/settings.py` | `get_setting()`, `set_setting()`, `get_all_settings()` — key-value per user | +| `services/tag_suggestions.py` | `/api/notes/suggest-tags` — LLM-generated tag suggestions for note body | +| `services/access.py` | Permission resolution for all shared resources | +| `services/sharing.py` | Create/revoke shares; list shares; `get_shared_with_me()` | +| `services/groups.py` | Group CRUD; membership management | +| `services/notifications.py` | Create/read/mark-read in-app notifications | +| `services/task_logs.py` | Append/list/delete work log entries on tasks | | `services/briefing_pipeline.py` | Two-lane parallel gather → LLM synthesis → `GenerationBuffer` stream | | `services/briefing_scheduler.py` | APScheduler `BackgroundScheduler`; slots with catch-up logic; async-safe via `asyncio.create_task` | +| `services/briefing_conversations.py` | Briefing conversation persistence and history queries | +| `services/briefing_profile.py` | Per-user profile note that the assistant updates over time | | `services/research.py` | SearXNG research pipeline: 5 sub-queries → parallel fetch → synthesis; `search_images` for image category | | `services/caldav.py` | Full CalDAV event/todo lifecycle; synchronous library calls in asyncio executor | +| `services/calendar_sync.py` | Dead code — Radicale sync service; was trialled and removed | | `services/images.py` | `fetch_and_store_image()` — SHA-256 dedup, content-type validation, 5 MB cap | | `services/backup.py` | `export_full_backup()`, `export_user_backup()`, `restore_full_backup()` (version 2 with ID maps) | | `services/push.py` | VAPID key auto-generation; `send_push_notification()` fire-and-forget; 410 Gone cleanup | | `services/logging.py` | `log_audit`, `log_usage`, `log_error`, `start_log_retention_loop` (hourly cleanup) | +| `services/email.py` | SMTP email sending for password reset; reads `SMTP_*` env vars | | `services/weather.py` | Nominatim geocoding + Open-Meteo forecast + per-user DB cache | | `services/rss.py` | feedparser fetch; per-feed DB cache; prune-to-100 items | | `services/assist.py` | `build_assist_messages()` — section/whole-doc modes; project context injection | @@ -261,9 +296,11 @@ Permission resolution is centralised in `services/access.py`. `get_project_permi ## Authentication -Local username/password auth and OIDC/OAuth (PKCE). `services/oauth.py` handles discovery and `find_or_create_oauth_user`. On OAuth login: checks existing `oauth_sub` → matching email → creates new user. +**Session cookies** — `HttpOnly`, `SameSite=Lax`, optionally `Secure` (`SECURE_COOKIES` env var). Session includes `session_version`; mismatch with DB value (after password change) results in 401 and session clear. -Session cookies: `HttpOnly`, `SameSite=Lax`, optionally `Secure`. Session includes `session_version`; mismatch with DB value (after password change) results in 401 and session clear. +**Bearer token / API keys** — `Authorization: Bearer ` accepted by `_check_auth()` as an alternative to session cookies. The raw key is SHA-256 hashed and looked up via `services/api_keys.py`. Keys with `scope=read` are rejected on non-safe methods (`POST`/`PATCH`/`DELETE`). Used by Fable MCP and any external API consumers. + +**Local auth + OIDC/OAuth (PKCE)** — `services/oauth.py` handles discovery and `find_or_create_oauth_user`. On OAuth login: checks existing `oauth_sub` → matching email → creates new user. See [sso-oauth.md](sso-oauth.md) for provider-specific setup instructions. diff --git a/docs/development.md b/docs/development.md index ae21c2c..592ae86 100644 --- a/docs/development.md +++ b/docs/development.md @@ -117,7 +117,7 @@ Current migration sequence (all idempotent raw SQL): 0016 add_image_cache 0017 add_projects 0018 add_push_subscriptions -0019 add_note_parent_and_project_fields +0019 add_events (dead code — internal CalDAV/Radicale table; Radicale was removed) 0020 add_milestones 0021 add_task_logs 0022 add_note_versions_and_drafts @@ -125,6 +125,7 @@ Current migration sequence (all idempotent raw SQL): 0024 add_session_version 0025 add_sharing_and_notifications 0026 add_briefing_tables +0027 add_api_keys ``` **Important:** Do NOT use `op.create_table()` or `sa.Enum()` — SQLAlchemy's event system can fire `CREATE TYPE` even with `create_type=False`, causing failures on re-run. Always use raw SQL with `IF NOT EXISTS` / `DO $$ BEGIN ... EXCEPTION WHEN duplicate_object` guards.