|
|
|
@@ -12,7 +12,7 @@
|
|
|
|
|
> Include file-level details in the commit body when the change is non-trivial.
|
|
|
|
|
|
|
|
|
|
## Last Updated
|
|
|
|
|
2026-02-12 — Phase 5.3: Registration Control, User Management, Security Hardening
|
|
|
|
|
2026-02-13 — Phase 5.6: Assist Background-Task + Buffer Architecture
|
|
|
|
|
|
|
|
|
|
## Project Overview
|
|
|
|
|
Fabled Assistant is a self-hosted note-taking and task-tracking application with
|
|
|
|
@@ -66,6 +66,9 @@ for AI-assisted features.
|
|
|
|
|
tail the buffer and can reconnect mid-stream without data loss. Buffer has
|
|
|
|
|
`cancel_event` for user-initiated stop. Completed buffers are cleaned up after
|
|
|
|
|
60s grace period. Periodic DB flushes every 5s preserve partial content.
|
|
|
|
|
Both chat and AI Assist use this architecture — assist buffers are keyed by
|
|
|
|
|
`"assist:{user_id}"` (string keys) instead of conversation ID (int keys).
|
|
|
|
|
Assist generation has no DB persistence, no title generation, no cancellation.
|
|
|
|
|
- **SSE over WebSockets for LLM streaming:** SSE clients connect via
|
|
|
|
|
`GET /api/chat/conversations/:id/generation/stream` with `Last-Event-ID`
|
|
|
|
|
reconnection support. Frontend uses `fetch()` + `ReadableStream`. Simpler than
|
|
|
|
@@ -175,6 +178,18 @@ for AI-assisted features.
|
|
|
|
|
- `to_dict()` returns: `id`, `title`, `model`, `message_count`, `created_at`, `updated_at`
|
|
|
|
|
- `list_conversations()` uses a subquery for `message_count` instead of eager-loading all messages
|
|
|
|
|
|
|
|
|
|
### App Logs
|
|
|
|
|
- `id` (int PK), `category` (text NOT NULL — `audit`/`usage`/`error`),
|
|
|
|
|
`user_id` (int FK users ON DELETE SET NULL), `username` (text — denormalized),
|
|
|
|
|
`action` (text), `endpoint` (text), `method` (text), `status_code` (int),
|
|
|
|
|
`duration_ms` (real), `ip_address` (text), `details` (text — JSON-serialized),
|
|
|
|
|
`created_at` (timestamptz DEFAULT now())
|
|
|
|
|
- Single table with `category` field for all log types
|
|
|
|
|
- Denormalized `username` preserves attribution after user deletion
|
|
|
|
|
- `details` stores flexible JSON data (error tracebacks, audit metadata, etc.)
|
|
|
|
|
- Indexes: `category`, `user_id`, `created_at`, composite `(category, created_at DESC)`
|
|
|
|
|
- Automatic retention cleanup via hourly asyncio task (configurable `LOG_RETENTION_DAYS`, default 90)
|
|
|
|
|
|
|
|
|
|
### Messages
|
|
|
|
|
- `id` (int PK), `conversation_id` (FK to conversations, CASCADE), `role` (str: system/user/assistant),
|
|
|
|
|
`content` (str), `status` (str, default `'complete'` — `complete`/`generating`/`error`),
|
|
|
|
@@ -203,7 +218,8 @@ fabledassistant/
|
|
|
|
|
│ ├── 0006_add_settings_table.py # Settings key-value table
|
|
|
|
|
│ ├── 0007_add_title_and_updated_at_indexes.py # B-tree indexes on notes.title and conversations.updated_at
|
|
|
|
|
│ ├── 0008_add_users_and_user_id.py # Users table, user_id FKs on notes/conversations/settings, composite PK for settings
|
|
|
|
|
│ └── 0009_add_message_status.py # Add status column to messages table (default 'complete')
|
|
|
|
|
│ ├── 0009_add_message_status.py # Add status column to messages table (default 'complete')
|
|
|
|
|
│ └── 0010_add_app_logs_table.py # App logs table for audit, usage, and error logging
|
|
|
|
|
├── src/
|
|
|
|
|
│ └── fabledassistant/
|
|
|
|
|
│ ├── __init__.py
|
|
|
|
@@ -215,7 +231,8 @@ fabledassistant/
|
|
|
|
|
│ │ ├── user.py # User model (id, username, email, password_hash, role, created_at)
|
|
|
|
|
│ │ ├── note.py # Note model (unified: id, title, body, tags[], parent_id, user_id, status, priority, due_date, timestamps)
|
|
|
|
|
│ │ ├── conversation.py # Conversation + Message models with user_id
|
|
|
|
|
│ │ └── setting.py # Setting model (composite PK: user_id + key, value TEXT)
|
|
|
|
|
│ │ ├── setting.py # Setting model (composite PK: user_id + key, value TEXT)
|
|
|
|
|
│ │ └── app_log.py # AppLog model (id, category, user_id, username, action, endpoint, method, status_code, duration_ms, ip_address, details, created_at)
|
|
|
|
|
│ ├── routes/
|
|
|
|
|
│ │ ├── __init__.py
|
|
|
|
|
│ │ ├── api.py # /api blueprint with /health endpoint (public)
|
|
|
|
@@ -231,9 +248,12 @@ fabledassistant/
|
|
|
|
|
│ │ ├── notes.py # CRUD with user_id isolation, is_task filter, convert, backlinks, search_notes_for_context
|
|
|
|
|
│ │ ├── llm.py # Ollama interaction: build_context with user_id, streaming, URL fetching
|
|
|
|
|
│ │ ├── chat.py # Conversation CRUD with user_id isolation, add_message, save/summarize as note (LLM-titled, chat-tagged)
|
|
|
|
|
│ │ ├── generation_buffer.py # In-memory SSE event buffer with cancel_event, reconnect support, auto-cleanup
|
|
|
|
|
│ │ ├── generation_task.py # Background asyncio task: streams LLM into buffer, periodic DB flush, LLM title generation
|
|
|
|
|
│ │ └── settings.py # Settings CRUD with user_id isolation: get_setting, set_setting, set_settings_batch, get_all_settings
|
|
|
|
|
│ │ ├── generation_buffer.py # In-memory SSE event buffer with cancel_event, reconnect support, auto-cleanup; supports chat (int keys) and assist (string keys)
|
|
|
|
|
│ │ ├── generation_task.py # Background asyncio tasks: run_generation (chat, DB flush, titles) + run_assist_generation (lightweight, no DB)
|
|
|
|
|
│ │ ├── settings.py # Settings CRUD with user_id isolation: get_setting, set_setting, set_settings_batch, get_all_settings
|
|
|
|
|
│ │ ├── logging.py # App logging: log_audit, log_usage, log_error, get_logs, get_log_stats, delete_old_logs, start_log_retention_loop
|
|
|
|
|
│ │ ├── email.py # SMTP email service: get_smtp_config, is_smtp_configured, send_email, send_test_email
|
|
|
|
|
│ │ └── notifications.py # Notification service: notify_security_event, check_due_tasks, start_notification_loop
|
|
|
|
|
│ ├── utils/
|
|
|
|
|
│ │ ├── __init__.py
|
|
|
|
|
│ │ └── tags.py # extract_tags() — regex #tag extraction, skips code fences
|
|
|
|
@@ -248,11 +268,11 @@ fabledassistant/
|
|
|
|
|
│ ├── assets/
|
|
|
|
|
│ │ └── theme.css # CSS custom properties: light/dark themes, body reset, design tokens, responsive breakpoints (480/768/1024), utility classes (.hide-mobile/.hide-desktop), mobile touch targets
|
|
|
|
|
│ ├── api/
|
|
|
|
|
│ │ └── client.ts # ApiError class, apiGet/apiPost/apiPut/apiPatch/apiDelete + apiSSEStream (EventSource-based), auto 401→login redirect
|
|
|
|
|
│ │ └── client.ts # ApiError class, apiGet/apiPost/apiPut/apiPatch/apiDelete + apiSSEStream (fetch+ReadableStream), apiStreamPost (legacy), auto 401→login redirect
|
|
|
|
|
│ ├── composables/
|
|
|
|
|
│ │ ├── useTheme.ts # Theme toggle, localStorage, prefers-color-scheme
|
|
|
|
|
│ │ ├── useAutocomplete.ts # Legacy textarea autocomplete (replaced by Tiptap suggestion extensions)
|
|
|
|
|
│ │ └── useAssist.ts # AI Assist composable: section parsing, target selection, LLM streaming, accept/reject; watches body ref for auto-sync
|
|
|
|
|
│ │ └── useAssist.ts # AI Assist composable: section parsing, target selection, two-step POST+SSE streaming (apiPost → apiSSEStream), accept/reject; watches body ref for auto-sync
|
|
|
|
|
│ ├── stores/
|
|
|
|
|
│ │ ├── auth.ts # Auth state: user, isAuthenticated, isAdmin, login/register/logout/checkAuth
|
|
|
|
|
│ │ ├── notes.ts # CRUD + tag filter, resolveTitle, convertToTask, convertToNote, fetchBacklinks, fetchAllTags (with toast errors)
|
|
|
|
@@ -290,7 +310,8 @@ fabledassistant/
|
|
|
|
|
│ │ ├── TaskEditorView.vue # Create/edit task: Tiptap editor, sticky toolbar, AI Assist panel, Ctrl+S, dirty guard
|
|
|
|
|
│ │ └── TaskViewerView.vue # Task detail: rendered markdown, badges, convert-to-note, backlinks
|
|
|
|
|
│ ├── components/
|
|
|
|
|
│ │ ├── AppHeader.vue # Nav bar: brand, nav links, status indicator, theme toggle, user info + logout, hamburger menu (mobile)
|
|
|
|
|
│ │ ├── LogsView.vue # Admin log viewer: stats summary, category/search/date filters, paginated table with expandable detail rows
|
|
|
|
|
│ │ ├── AppHeader.vue # Nav bar: brand, nav links (incl. admin Logs), status indicator, theme toggle, user info + logout, hamburger menu (mobile)
|
|
|
|
|
│ │ ├── ChatPanel.vue # Slide-out chat panel (right side overlay, receives contextNoteId prop), bubble-style messages, floating dark input, note picker, context pills with promote/exclude
|
|
|
|
|
│ │ ├── ChatMessage.vue # Message bubble: markdown rendering, configurable assistant name label, "Save as Note" action on assistant messages
|
|
|
|
|
│ │ ├── NoteCard.vue # Card with rendered markdown preview (v-html), TagPill, tag-click emit
|
|
|
|
@@ -305,7 +326,7 @@ fabledassistant/
|
|
|
|
|
│ │ ├── PaginationBar.vue # Prev/next + page numbers
|
|
|
|
|
│ │ └── ToastNotification.vue # Fixed-position toast container with close button, warning support
|
|
|
|
|
│ └── router/
|
|
|
|
|
│ └── index.ts # Routes: /, /login, /register, /notes/*, /tasks/*, /chat, /chat/:id, /settings, /admin/users; beforeEach auth guard
|
|
|
|
|
│ └── index.ts # Routes: /, /login, /register, /notes/*, /tasks/*, /chat, /chat/:id, /settings, /admin/users, /admin/logs; beforeEach auth guard
|
|
|
|
|
└── public/
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
@@ -326,6 +347,8 @@ fabledassistant/
|
|
|
|
|
| DELETE | `/api/admin/users/:id` | Delete a user (admin only, cannot delete self) |
|
|
|
|
|
| GET | `/api/admin/registration` | Get registration open/closed status (admin only) |
|
|
|
|
|
| PUT | `/api/admin/registration` | Toggle registration open/closed (admin only, body: `{open: bool}`) |
|
|
|
|
|
| GET | `/api/admin/logs` | List log entries (admin only, params: `category`, `user_id`, `search`, `date_from`, `date_to`, `limit`, `offset`) |
|
|
|
|
|
| GET | `/api/admin/logs/stats` | Get log category counts (admin only) |
|
|
|
|
|
| GET | `/api/notes` | List notes (params: `q`, `tag`, `sort`, `order`, `limit`, `offset`; defaults to `is_task=false` — plain notes only; `?is_task=true` for tasks, `?all=true` for everything) |
|
|
|
|
|
| POST | `/api/notes` | Create note (body: `{title, body, status?, priority?, due_date?}` — tags auto-extracted) |
|
|
|
|
|
| GET | `/api/notes/tags` | List all tags from notes table (param: `q` for filter) |
|
|
|
|
@@ -337,6 +360,8 @@ fabledassistant/
|
|
|
|
|
| POST | `/api/notes/:id/convert-to-task` | Set `status='todo'`, `priority='none'` on note (returns 200) |
|
|
|
|
|
| POST | `/api/notes/:id/convert-to-note` | Clear `status`, `priority`, `due_date` from note (returns 200) |
|
|
|
|
|
| GET | `/api/notes/:id/backlinks` | List notes/tasks that reference this note via wikilinks |
|
|
|
|
|
| POST | `/api/notes/assist` | Launch background assist generation, return 202 (body: `{body, target_section, instruction}`; 409 if already running) |
|
|
|
|
|
| GET | `/api/notes/assist/stream` | SSE endpoint tailing assist generation buffer; supports `Last-Event-ID` reconnection; emits `chunk`, `done`, `error` events with 15s keepalives |
|
|
|
|
|
| GET | `/api/tasks` | List tasks (params: `q`, `tag`, `status`, `priority`, `sort`, `order`, `limit`, `offset`) — queries notes where `status IS NOT NULL` |
|
|
|
|
|
| POST | `/api/tasks` | Create task (body: `{title, body, status?, priority?, due_date?}` — accepts `description` as fallback for `body`) |
|
|
|
|
|
| GET | `/api/tasks/:id` | Get single task |
|
|
|
|
@@ -357,6 +382,9 @@ fabledassistant/
|
|
|
|
|
| GET | `/api/chat/models` | List available Ollama models |
|
|
|
|
|
| POST | `/api/chat/models/pull` | Pull/download a model from Ollama via SSE streaming progress (body: `{model}`, response: SSE with `{status, completed, total}` events) |
|
|
|
|
|
| POST | `/api/chat/models/delete` | Delete a model from Ollama (body: `{model}`) |
|
|
|
|
|
| GET | `/api/admin/smtp` | Get SMTP config (password masked) (admin only) |
|
|
|
|
|
| PUT | `/api/admin/smtp` | Save SMTP config to admin settings (admin only, body: `{smtp_host, smtp_port, ...}`) |
|
|
|
|
|
| POST | `/api/admin/smtp/test` | Send test email (admin only, body: `{recipient}`) |
|
|
|
|
|
| GET | `/api/settings` | Get all app settings as `{key: value}` dict |
|
|
|
|
|
| PUT | `/api/settings` | Update settings (body: `{key: value, ...}`) |
|
|
|
|
|
|
|
|
|
@@ -369,7 +397,7 @@ container startup.
|
|
|
|
|
|
|
|
|
|
### Migration Chain
|
|
|
|
|
```
|
|
|
|
|
0001_create_notes_table.py → 0002_create_tasks_table.py → 0003_task_note_companion.py → 0004_merge_tasks_into_notes.py → 0005_add_chat_tables.py → 0006_add_settings_table.py → 0007_add_title_and_updated_at_indexes.py → 0008_add_users_and_user_id.py → 0009_add_message_status.py
|
|
|
|
|
0001_create_notes_table.py → 0002_create_tasks_table.py → 0003_task_note_companion.py → 0004_merge_tasks_into_notes.py → 0005_add_chat_tables.py → 0006_add_settings_table.py → 0007_add_title_and_updated_at_indexes.py → 0008_add_users_and_user_id.py → 0009_add_message_status.py → 0010_add_app_logs_table.py
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
### How Migrations Run
|
|
|
|
@@ -609,6 +637,34 @@ When adding a new migration, follow these conventions:
|
|
|
|
|
- [x] **Password change:** `PUT /api/auth/password` endpoint + Settings UI section
|
|
|
|
|
- [x] **Production deployment docs:** README documents reverse proxy, rate limiting, CSP headers, SECRET_KEY, registration behavior
|
|
|
|
|
|
|
|
|
|
### Phase 5.4 — Application Logging System ✓
|
|
|
|
|
- [x] **Single `app_logs` table:** Unified audit/usage/error logging with `category` field
|
|
|
|
|
- [x] **Usage logging middleware:** `after_request` logs all `/api/*` requests with user, endpoint, method, status, duration (skips `/api/admin/logs` to avoid recursion)
|
|
|
|
|
- [x] **Error logging:** `handle_500` captures error type, message, and traceback
|
|
|
|
|
- [x] **Audit logging:** Security events logged in auth routes (register, login, login_failed, logout, password_change) and admin routes (backup, restore, user_delete, registration_toggle)
|
|
|
|
|
- [x] **Denormalized username:** Preserves attribution after user deletion (`user_id` FK uses `ON DELETE SET NULL`)
|
|
|
|
|
- [x] **Admin log viewer:** `/admin/logs` with stats summary, category/search/date filters, paginated table with expandable JSON detail rows
|
|
|
|
|
- [x] **Log stats endpoint:** `GET /api/admin/logs/stats` returns category counts
|
|
|
|
|
- [x] **Configurable retention:** `LOG_RETENTION_DAYS` env var (default 90), hourly asyncio cleanup task
|
|
|
|
|
- [x] **Config:** `LOG_RETENTION_DAYS` added to `Config` class
|
|
|
|
|
|
|
|
|
|
### Phase 5.5 — SMTP Email Notifications ✓
|
|
|
|
|
- [x] **SMTP email service:** `aiosmtplib` for async email sending, supports STARTTLS (587) and implicit TLS (465)
|
|
|
|
|
- [x] **SMTP config in DB:** Admin configures SMTP via Settings UI, stored in `settings` table; env var fallbacks for Docker Swarm bootstrap
|
|
|
|
|
- [x] **Admin SMTP endpoints:** `GET/PUT /api/admin/smtp` for config management, `POST /api/admin/smtp/test` for sending test emails
|
|
|
|
|
- [x] **Security alert notifications:** Fire-and-forget `asyncio.create_task()` on login, failed login, logout, password change
|
|
|
|
|
- [x] **Task due date reminders:** Hourly background loop checks for due/overdue tasks, sends grouped email per user; dedup via `app_logs` check
|
|
|
|
|
- [x] **Per-user notification preferences:** `notify_task_reminders` and `notify_security_alerts` settings (default enabled)
|
|
|
|
|
- [x] **Settings UI:** Notifications section for all users (checkbox toggles), SMTP section for admin (2-column grid + test email)
|
|
|
|
|
|
|
|
|
|
### Phase 5.6 — Assist Background-Task + Buffer Architecture ✓
|
|
|
|
|
- [x] **Assist buffer helpers:** `create_assist_buffer()`, `get_assist_buffer()`, `remove_assist_buffer()` using `"assist:{user_id}"` string keys in shared `_buffers` registry
|
|
|
|
|
- [x] **`run_assist_generation()`:** Lightweight background task — streams from Ollama into buffer, no DB persistence, no title generation, no cancellation
|
|
|
|
|
- [x] **Two-step assist API:** `POST /api/notes/assist` returns 202 + launches background task; `GET /api/notes/assist/stream` SSE endpoint tails buffer with 15s keepalives and `Last-Event-ID` reconnection
|
|
|
|
|
- [x] **409 conflict guard:** Rejects concurrent assist requests per user
|
|
|
|
|
- [x] **Frontend two-step pattern:** `useAssist.ts` calls `apiPost()` then `apiSSEStream()` with named event mapping (`chunk`/`done`/`error`); tracks stream handle for cleanup
|
|
|
|
|
- [x] **Fixes `NS_ERROR_NET_PARTIAL_TRANSFER`:** Keepalive pings prevent browser/Hypercorn from closing connection during gaps between LLM chunks
|
|
|
|
|
|
|
|
|
|
### Future / Stretch
|
|
|
|
|
- Tagging/labeling system with LLM-suggested tags
|
|
|
|
|
- Calendar/timeline view for tasks
|
|
|
|
@@ -624,7 +680,7 @@ When adding a new migration, follow these conventions:
|
|
|
|
|
- To reset database: `docker compose down -v && docker compose up --build`
|
|
|
|
|
|
|
|
|
|
## Current Status
|
|
|
|
|
**Phase:** Phase 5.3 complete. Registration Control, User Management, Security Hardening.
|
|
|
|
|
**Phase:** Phase 5.6 complete. Assist Background-Task + Buffer Architecture.
|
|
|
|
|
- Full note-taking and task-tracking CRUD with markdown, wikilinks, backlinks, tags
|
|
|
|
|
- **Tiptap WYSIWYG editor** with inline formatting preview, markdown round-trip, paste handling
|
|
|
|
|
- **Tag/wikilink autocomplete** via `@tiptap/suggestion` with heading disambiguation
|
|
|
|
@@ -632,7 +688,7 @@ When adding a new migration, follow these conventions:
|
|
|
|
|
- **Multi-user authentication** with session cookies, bcrypt passwords, admin role
|
|
|
|
|
- **Per-user data isolation** across notes, conversations, and settings
|
|
|
|
|
- LLM chat via Ollama with background generation task + SSE streaming
|
|
|
|
|
- **AI Assist panel** pinned to bottom 1/3 of editor viewport with auto-syncing sections
|
|
|
|
|
- **AI Assist panel** pinned to bottom 1/3 of editor viewport with auto-syncing sections; uses background-task + buffer architecture with keepalive SSE (same pattern as chat)
|
|
|
|
|
- Note-aware context: auto-includes current note + searches related notes by keyword
|
|
|
|
|
- Context pills with promote/exclude controls; note picker in chat input
|
|
|
|
|
- Save assistant messages as notes (LLM-titled, chat-tagged); summarize conversations as notes
|
|
|
|
@@ -644,4 +700,10 @@ When adding a new migration, follow these conventions:
|
|
|
|
|
- **App-wide layout fix**: navbar always visible, all views fit within viewport
|
|
|
|
|
- **Responsive design**: hamburger menu, mobile touch targets, responsive breakpoints
|
|
|
|
|
- **Docker Swarm production stack** with secrets, network isolation, health checks, resource limits
|
|
|
|
|
- **Application logging**: audit (security events), usage (API requests), error (unhandled exceptions)
|
|
|
|
|
- **Admin log viewer**: `/admin/logs` with stats, filters, pagination, expandable detail rows
|
|
|
|
|
- **Automatic log retention**: configurable via `LOG_RETENTION_DAYS` (default 90 days)
|
|
|
|
|
- **SMTP email notifications**: security alerts (login/logout/failed login/password change), task due date reminders
|
|
|
|
|
- **Admin SMTP configuration**: Settings UI with test email, DB-stored config with env var fallbacks
|
|
|
|
|
- **Per-user notification preferences**: toggle task reminders and security alerts independently
|
|
|
|
|
- Dark/light theme with CSS custom properties and design tokens
|
|
|
|
|