Backend efficiency & consistency pass
- Rewrite get_all_tags() with SQL unnest instead of loading all notes - Consolidate convert_note_to_task/convert_task_to_note to single-session ops - Add search_notes_for_context() with single OR-keyword query for build_context() - Drop selectinload from list_conversations(), use correlated subquery for message_count - Add set_settings_batch() for single-transaction multi-setting upserts - Extract get_installed_models() shared helper into services/llm.py - Delete services/tasks.py pass-through wrapper; routes/tasks.py imports from services.notes - Add B-tree indexes on notes.title and conversations.updated_at (migration 0007) - Add logging to services/notes.py, services/chat.py, services/settings.py - Safe Conversation.to_dict() when messages relationship is not loaded Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
+28
-14
@@ -12,7 +12,7 @@
|
||||
> Include file-level details in the commit body when the change is non-trivial.
|
||||
|
||||
## Last Updated
|
||||
2026-02-10 — Phase 4.7: Styling consistency pass — design tokens, header alignment, standardized UI
|
||||
2026-02-11 — Phase 4.8: Backend efficiency & consistency pass
|
||||
|
||||
## Project Overview
|
||||
Fabled Assistant is a self-hosted note-taking and task-tracking application with
|
||||
@@ -127,11 +127,11 @@ for AI-assisted features.
|
||||
- Auto-create via `get_or_create_note_by_title()` for wikilink clicks
|
||||
- `to_dict()` returns: `id`, `title`, `body`, `tags`, `parent_id`, `status`,
|
||||
`priority`, `due_date`, `is_task`, `created_at`, `updated_at`
|
||||
- Indexes: GIN on tags, B-tree on status
|
||||
- Indexes: GIN on tags, B-tree on status, B-tree on title
|
||||
|
||||
### Task ≡ Note with task attributes
|
||||
- No separate tasks table. The `services/tasks.py` module is a thin wrapper
|
||||
around `services/notes.py` that passes `is_task=True` for listing and
|
||||
- No separate tasks table. `routes/tasks.py` imports directly from
|
||||
`services/notes.py` with `is_task=True` for listing and
|
||||
defaults `status='todo'`, `priority='none'` for creation.
|
||||
- "Convert to task" = `update_note(id, status='todo', priority='none')`
|
||||
- "Convert to note" = `update_note(id, status=None, priority=None, due_date=None)`
|
||||
@@ -140,13 +140,15 @@ for AI-assisted features.
|
||||
### Settings
|
||||
- `key` (text PK), `value` (text)
|
||||
- Key-value store for app-wide settings (assistant name, default model, etc.)
|
||||
- CRUD via `services/settings.py`: `get_setting(key, default)`, `set_setting(key, value)`, `get_all_settings()`
|
||||
- CRUD via `services/settings.py`: `get_setting(key, default)`, `set_setting(key, value)`, `set_settings_batch(dict)`, `get_all_settings()`
|
||||
|
||||
### Conversations
|
||||
- `id` (int PK), `title` (str), `model` (str), `created_at`, `updated_at`
|
||||
- Has many messages (cascade delete)
|
||||
- Title auto-generated from first user message if not set
|
||||
- Index on `updated_at` for list ordering
|
||||
- `to_dict()` returns: `id`, `title`, `model`, `message_count`, `created_at`, `updated_at`
|
||||
- `list_conversations()` uses a subquery for `message_count` instead of eager-loading all messages
|
||||
|
||||
### Messages
|
||||
- `id` (int PK), `conversation_id` (FK to conversations, CASCADE), `role` (str: system/user/assistant),
|
||||
@@ -171,7 +173,8 @@ fabledassistant/
|
||||
│ ├── 0003_task_note_companion.py # Data migration: create companion notes for existing tasks
|
||||
│ ├── 0004_merge_tasks_into_notes.py # Add task columns to notes, migrate data, drop tasks table
|
||||
│ ├── 0005_add_chat_tables.py # Conversations + messages tables with FKs and indexes
|
||||
│ └── 0006_add_settings_table.py # Settings key-value table
|
||||
│ ├── 0006_add_settings_table.py # Settings key-value table
|
||||
│ └── 0007_add_title_and_updated_at_indexes.py # B-tree indexes on notes.title and conversations.updated_at
|
||||
├── src/
|
||||
│ └── fabledassistant/
|
||||
│ ├── __init__.py
|
||||
@@ -187,14 +190,13 @@ fabledassistant/
|
||||
│ │ ├── api.py # /api blueprint with /health endpoint
|
||||
│ │ ├── chat.py # /api/chat blueprint: conversations CRUD, SSE message streaming, save/summarize as note, models list/pull/delete, status check
|
||||
│ │ ├── notes.py # /api/notes CRUD + /by-title + /resolve-title + /convert-to-task + /convert-to-note + /backlinks
|
||||
│ │ ├── tasks.py # /api/tasks CRUD + PATCH status (thin wrappers, accepts body not description)
|
||||
│ │ ├── tasks.py # /api/tasks CRUD + PATCH status (imports directly from services.notes, accepts body not description)
|
||||
│ │ └── settings.py # /api/settings GET/PUT — app-wide settings
|
||||
│ ├── services/
|
||||
│ │ ├── notes.py # CRUD, is_task filter, status/priority filters, convert_note_to_task, convert_task_to_note, get_backlinks
|
||||
│ │ ├── tasks.py # Thin wrappers around notes.py (create_task, list_tasks with is_task=True, etc.)
|
||||
│ │ ├── llm.py # Ollama interaction: ensure_model, stream_chat, generate_completion, fetch_url_content, build_context (keyword extraction, note search with 5 keywords + 2000-char previews, exclude_note_ids support, returns (messages, context_meta) tuple, URL fetching, configurable assistant name)
|
||||
│ │ ├── chat.py # Conversation CRUD, add_message, save_response_as_note, summarize_conversation_as_note
|
||||
│ │ └── settings.py # Settings CRUD: get_setting, set_setting, get_all_settings
|
||||
│ │ ├── notes.py # CRUD, is_task filter, status/priority filters, convert_note_to_task, convert_task_to_note, get_backlinks, search_notes_for_context (OR-keyword search), get_all_tags (SQL unnest)
|
||||
│ │ ├── llm.py # Ollama interaction: get_installed_models (shared helper), ensure_model, stream_chat, generate_completion, fetch_url_content, build_context (keyword extraction, single OR-query note search, exclude_note_ids support, returns (messages, context_meta) tuple, URL fetching, configurable assistant name)
|
||||
│ │ ├── chat.py # Conversation CRUD (list uses subquery count), add_message, save_response_as_note, summarize_conversation_as_note
|
||||
│ │ └── settings.py # Settings CRUD: get_setting, set_setting, set_settings_batch, get_all_settings
|
||||
│ ├── utils/
|
||||
│ │ ├── __init__.py
|
||||
│ │ └── tags.py # extract_tags() — regex #tag extraction, skips code fences
|
||||
@@ -301,7 +303,7 @@ container startup.
|
||||
|
||||
### Migration Chain
|
||||
```
|
||||
0001_create_notes_table.py → 0002_create_tasks_table.py → 0003_task_note_companion.py → 0004_merge_tasks_into_notes.py → 0005_add_chat_tables.py → 0006_add_settings_table.py
|
||||
0001_create_notes_table.py → 0002_create_tasks_table.py → 0003_task_note_companion.py → 0004_merge_tasks_into_notes.py → 0005_add_chat_tables.py → 0006_add_settings_table.py → 0007_add_title_and_updated_at_indexes.py
|
||||
```
|
||||
|
||||
### How Migrations Run
|
||||
@@ -481,6 +483,18 @@ When adding a new migration, follow these conventions:
|
||||
- [x] **Focus-visible states:** Global `focus-visible` rule with `--focus-ring` shadow on inputs, textareas, selects, buttons
|
||||
- [x] **Modal overlays:** Replaced hardcoded `rgba(0,0,0,0.5)` with `var(--color-overlay)` across all components
|
||||
|
||||
### Phase 4.8 — Backend Efficiency & Consistency Pass ✓
|
||||
- [x] **`get_all_tags()` SQL rewrite:** Replaced O(n) Python-side tag extraction with single `SELECT DISTINCT unnest(tags)` query
|
||||
- [x] **`list_conversations()` subquery count:** Replaced `selectinload(Conversation.messages)` with correlated subquery for `message_count`, avoiding loading all messages for list view
|
||||
- [x] **Removed `services/tasks.py`:** Eliminated 64-line pass-through service; `routes/tasks.py` imports directly from `services.notes`
|
||||
- [x] **Consolidated convert functions:** `convert_note_to_task()` and `convert_task_to_note()` now use single session (was 3 DB round trips)
|
||||
- [x] **Batch settings updates:** New `set_settings_batch()` does all upserts in single transaction (was one transaction per setting)
|
||||
- [x] **Shared `get_installed_models()`:** Extracted duplicate Ollama model-listing logic from 3 locations into `services/llm.py`
|
||||
- [x] **`search_notes_for_context()`:** New function does single OR-keyword query (was up to 5 sequential `list_notes()` calls each with count query)
|
||||
- [x] **Logging:** Added `logger = logging.getLogger(__name__)` to `services/notes.py`, `services/chat.py`, `services/settings.py`
|
||||
- [x] **Database indexes:** Added B-tree indexes on `notes.title` (for `get_note_by_title` + backlinks) and `conversations.updated_at` (for list ordering)
|
||||
- [x] **Safe `Conversation.to_dict()`:** Handles case where messages relationship is not loaded
|
||||
|
||||
### Phase 5 — Polish & Production Hardening
|
||||
- [ ] Authentication (single-user or multi-user, TBD)
|
||||
- [ ] Docker Swarm production stack (secrets, volumes, networking)
|
||||
@@ -506,7 +520,7 @@ When adding a new migration, follow these conventions:
|
||||
- Authentication model: single-user (password-only) vs multi-user?
|
||||
|
||||
## Current Status
|
||||
**Phase:** Phase 4.7 complete. Styling consistency pass with design tokens and standardized UI.
|
||||
**Phase:** Phase 4.8 complete. Backend efficiency & consistency pass.
|
||||
- Full note-taking and task-tracking CRUD with markdown, wikilinks, backlinks, tags
|
||||
- Unified note/task model (a task is a note with `status IS NOT NULL`)
|
||||
- LLM chat via Ollama with SSE streaming responses
|
||||
|
||||
Reference in New Issue
Block a user