docs: update architecture and features for calendar + RAG scoping

- architecture.md: add Event model, services/events.py, routes/events.py,
  CalendarView.vue, EventSlideOver.vue; update Conversation and Project
  data model tables with new columns; update RAG Pipeline section with
  three-value scope system, search_projects/set_rag_scope tools, and
  project summary background job; fix dead-code note for models/event.py;
  update execute_tool() signature docs; add services/projects.py entry
- features.md: replace CalDAV section with full Calendar section covering
  internal store, AI tools, HomeView widget, and optional CalDAV sync;
  update AI Chat scope chip description; remove done "Calendar view" item
  from roadmap

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
2026-03-25 18:16:41 -04:00
parent ebc79b34f9
commit 87c55691fb
2 changed files with 44 additions and 15 deletions
+33 -11
View File
@@ -137,14 +137,14 @@ Composite PK `(user_id, key)`. Per-user key-value store. CRUD via `services/sett
### Conversations / Messages ### Conversations / Messages
`conversations`: `id`, `title`, `model`, `user_id`, `conversation_type` (`chat`/`briefing`/`mcp`), `briefing_date`, `created_at`, `updated_at`. `conversations`: `id`, `title`, `model`, `user_id`, `conversation_type` (`chat`/`briefing`/`mcp`), `briefing_date`, `rag_project_id` (nullable int — RAG scope: NULL=orphan-only, -1=all, positive=project), `created_at`, `updated_at`.
`messages`: `id`, `conversation_id` FK CASCADE, `role` (`user`/`assistant`), `content`, `status` (`done`/`generating`), `created_at`. `messages`: `id`, `conversation_id` FK CASCADE, `role` (`user`/`assistant`), `content`, `status` (`done`/`generating`), `created_at`.
Title auto-generated by LLM on first exchange, re-generated every 10th message. Title auto-generated by LLM on first exchange, re-generated every 10th message.
### Projects / Milestones ### Projects / Milestones
`projects`: `id`, `user_id`, `title`, `description`, `goal`, `status` (`active`/`completed`/`archived`), `color`, timestamps. `projects`: `id`, `user_id`, `title`, `description`, `goal`, `status` (`active`/`completed`/`archived`), `color`, `auto_summary` (nullable text — LLM-generated summary for `search_projects` scoring), `summary_updated_at` (nullable timestamptz), timestamps.
`milestones`: `id`, `user_id`, `project_id` FK CASCADE, `title`, `description`, `status`, `order_index`, timestamps. `milestones`: `id`, `user_id`, `project_id` FK CASCADE, `title`, `description`, `status`, `order_index`, timestamps.
### Sharing & Access ### Sharing & Access
@@ -184,7 +184,7 @@ Permission resolution is centralised in `services/access.py`. `get_project_permi
| `models/conversation.py` | `Conversation` + `Message` models | | `models/conversation.py` | `Conversation` + `Message` models |
| `models/app_log.py` | `AppLog` with `category`, denormalised `username`, `details` JSONB | | `models/app_log.py` | `AppLog` with `category`, denormalised `username`, `details` JSONB |
| `models/api_key.py` | `ApiKey`: `id`, `user_id`, `prefix`, `key_hash` (SHA-256), `name`, `scope` (`read`/`write`), `created_at`, `last_used_at` | | `models/api_key.py` | `ApiKey`: `id`, `user_id`, `prefix`, `key_hash` (SHA-256), `name`, `scope` (`read`/`write`), `created_at`, `last_used_at` |
| `models/event.py` | Dead code — CalDAV/Radicale internal events table. Radicale was trialled and removed; `calendar_sync.py` is also dead. | | `models/event.py` | Internal events store (`Event` model: `id`, `user_id`, `title`, `description`, `start_dt`, `end_dt`, `all_day`, `location`, `caldav_uid` nullable, `color`, timestamps) |
| `routes/api.py` | `/api` blueprint; `GET /api/health` (public) | | `routes/api.py` | `/api` blueprint; `GET /api/health` (public) |
| `routes/auth.py` | Register, login, logout, me, password/email change, password reset, invite registration, OAuth login+callback; rate limiting; `LOCAL_AUTH_ENABLED` guards | | `routes/auth.py` | Register, login, logout, me, password/email change, password reset, invite registration, OAuth login+callback; rate limiting; `LOCAL_AUTH_ENABLED` guards |
| `routes/admin.py` | Backup, restore, user management, registration toggle, invitations, base URL, SMTP (admin only) | | `routes/admin.py` | Backup, restore, user management, registration toggle, invitations, base URL, SMTP (admin only) |
@@ -213,8 +213,9 @@ Permission resolution is centralised in `services/access.py`. `get_project_permi
| `services/llm.py` | `build_context()`, RAG injection, history summarisation, `stream_chat_with_tools()`, URL fetching, SSRF guard | | `services/llm.py` | `build_context()`, RAG injection, history summarisation, `stream_chat_with_tools()`, URL fetching, SSRF guard |
| `services/generation_task.py` | `run_generation()` — full chat pipeline: intent routing, tool loop, SSE fan-out, push notification; `run_assist_generation()` | | `services/generation_task.py` | `run_generation()` — full chat pipeline: intent routing, tool loop, SSE fan-out, push notification; `run_assist_generation()` |
| `services/intent.py` | `classify_intent()` — fast non-streaming LLM call; intent skip heuristic; `_PRIOR_WORK_REFS` fast-path | | `services/intent.py` | `classify_intent()` — fast non-streaming LLM call; intent skip heuristic; `_PRIOR_WORK_REFS` fast-path |
| `services/tools.py` | All LLM tool definitions + `execute_tool()` dispatcher; duplicate guards; `_resolve_project()` 4-step lookup | | `services/tools.py` | All LLM tool definitions + `execute_tool(user_id, tool_name, arguments, conv_id=None, workspace_project_id=None)` dispatcher; duplicate guards; `_resolve_project()` 4-step lookup; `search_projects` and `set_rag_scope` tools |
| `services/embeddings.py` | `upsert_note_embedding()`, `semantic_search_notes()` (pgvector cosine similarity) | | `services/projects.py` | Project CRUD + `generate_project_summary()` (Ollama, fire-and-forget) + `backfill_project_summaries()` (startup) |
| `services/embeddings.py` | `upsert_note_embedding()`, `semantic_search_notes(orphan_only=False)` (pgvector cosine similarity) |
| `services/generation_buffer.py` | In-memory SSE event buffer; `cancel_event`; 60s cleanup; supports both chat (int keys) and assist (string keys) | | `services/generation_buffer.py` | In-memory SSE event buffer; `cancel_event`; 60s cleanup; supports both chat (int keys) and assist (string keys) |
| `services/notes.py` | Note CRUD, wikilink resolution, backlink queries, tag management | | `services/notes.py` | Note CRUD, wikilink resolution, backlink queries, tag management |
| `services/note_versions.py` | Version snapshot on save; restore-from-version; diff metadata | | `services/note_versions.py` | Version snapshot on save; restore-from-version; diff metadata |
@@ -231,7 +232,9 @@ Permission resolution is centralised in `services/access.py`. `get_project_permi
| `services/briefing_conversations.py` | Briefing conversation persistence and history queries | | `services/briefing_conversations.py` | Briefing conversation persistence and history queries |
| `services/briefing_profile.py` | Per-user profile note that the assistant updates over time | | `services/briefing_profile.py` | Per-user profile note that the assistant updates over time |
| `services/research.py` | SearXNG research pipeline: 5 sub-queries → parallel fetch → synthesis; `search_images` for image category | | `services/research.py` | SearXNG research pipeline: 5 sub-queries → parallel fetch → synthesis; `search_images` for image category |
| `services/caldav.py` | Full CalDAV event/todo lifecycle; synchronous library calls in asyncio executor | | `services/events.py` | Internal events CRUD: `list_events`, `create_event`, `update_event`, `delete_event`, `get_event`; source of truth for all event LLM tools |
| `routes/events.py` | `/api/events` — event CRUD routes |
| `services/caldav.py` | Optional CalDAV sync — user-configured external server; syncs to/from internal store via `caldav_uid` FK; `is_caldav_configured()` guards tool activation |
| `services/calendar_sync.py` | Dead code — Radicale sync service; was trialled and removed | | `services/calendar_sync.py` | Dead code — Radicale sync service; was trialled and removed |
| `services/images.py` | `fetch_and_store_image()` — SHA-256 dedup, content-type validation, 5 MB cap | | `services/images.py` | `fetch_and_store_image()` — SHA-256 dedup, content-type validation, 5 MB cap |
| `services/backup.py` | `export_full_backup()`, `export_user_backup()`, `restore_full_backup()` (version 2 with ID maps) | | `services/backup.py` | `export_full_backup()`, `export_user_backup()`, `restore_full_backup()` (version 2 with ID maps) |
@@ -256,7 +259,9 @@ Permission resolution is centralised in `services/access.py`. `get_project_permi
| `stores/push.ts` | `isSupported`, `permission`, `isSubscribed`, `subscribe/unsubscribe` | | `stores/push.ts` | `isSupported`, `permission`, `isSubscribed`, `subscribe/unsubscribe` |
| `stores/notifications.ts` | `count`, `items`, `fetchCount`, `fetchAll`, `markRead`, `markAll` | | `stores/notifications.ts` | `count`, `items`, `fetchCount`, `fetchAll`, `markRead`, `markAll` |
| `views/HomeView.vue` | Chat-first dashboard; 6 task sections (Overdue → Other); 8 recent notes; inline streaming response | | `views/HomeView.vue` | Chat-first dashboard; 6 task sections (Overdue → Other); 8 recent notes; inline streaming response |
| `views/ChatView.vue` | `/chat` — SSE streaming; context sidebar (Suggested/In Context); note picker; model selector; message queue; bulk delete | | `views/ChatView.vue` | `/chat` — SSE streaming; context sidebar (Suggested/In Context); note picker; scope chip (RAG project scope pill above input); message queue; bulk delete |
| `views/CalendarView.vue` | `/calendar` — FullCalendar v6 month/week/day views; click to create event; click event to open EventSlideOver |
| `components/EventSlideOver.vue` | Reusable slide-over for event create/edit/delete; used in ToolCallCard, HomeView, CalendarView |
| `views/WorkspaceView.vue` | `/workspace/:id` — 3-panel (tasks/chat/notes); SSE tool-call watcher; conv persisted to localStorage | | `views/WorkspaceView.vue` | `/workspace/:id` — 3-panel (tasks/chat/notes); SSE tool-call watcher; conv persisted to localStorage |
| `views/GraphView.vue` | `/graph` — D3 force-directed; tag/note/project-hub nodes; physics panel; peek panel | | `views/GraphView.vue` | `/graph` — D3 force-directed; tag/note/project-hub nodes; physics panel; peek panel |
| `views/ProjectView.vue` | Kanban grouped by milestone; advance buttons; milestone management | | `views/ProjectView.vue` | Kanban grouped by milestone; advance buttons; milestone management |
@@ -318,7 +323,7 @@ If a tool is detected, the intent's one-sentence `ack` field is streamed as the
### Tool Loop ### Tool Loop
Multi-round tool loop (max 5 rounds). All implementations in `services/tools.py`; `execute_tool()` is the dispatcher. Multi-round tool loop (max 5 rounds). All implementations in `services/tools.py`; `execute_tool(user_id, tool_name, arguments, conv_id=None, workspace_project_id=None)` is the dispatcher. `conv_id` and `workspace_project_id` are threaded in from `run_generation()` so tools like `set_rag_scope` can write to the current conversation.
**Duplicate protection on `create_note` / `create_task`:** **Duplicate protection on `create_note` / `create_task`:**
1. Exact title match (case-insensitive) → hard block, redirect to `update_note` 1. Exact title match (case-insensitive) → hard block, redirect to `update_note`
@@ -352,11 +357,28 @@ Config: `IMAGE_CACHE_DIR` (default `/data/images`), `IMAGE_MAX_BYTES` (default 5
## RAG Pipeline ## RAG Pipeline
1. `semantic_search_notes()` — cosine similarity via pgvector, threshold configurable per call. 1. `semantic_search_notes()` — cosine similarity via pgvector, threshold configurable per call; accepts `orphan_only` and `project_id` scope flags.
2. Notes ≥ 0.60 similarity auto-injected into system prompt (up to 3, 800 chars each). 2. Notes ≥ 0.60 similarity auto-injected into system prompt (up to 3, 800 chars each).
3. Notes 0.450.60 surfaced in chat sidebar as "Suggested" (user clicks to include). 3. Notes 0.450.60 surfaced in chat sidebar as "Suggested" (user clicks to include).
4. Explicitly included notes delivered full-body. 4. Explicitly included notes delivered full-body.
5. Project-scoped RAG: `rag_project_id` param restricts all searches to a project's notes. 5. `excluded_note_ids` prevents the current note from being injected as its own context.
6. `excluded_note_ids` prevents the current note from being injected as its own context.
### RAG Scope (three-value system)
`conversations.rag_project_id` controls which notes are eligible for retrieval:
| Value | Behaviour |
|-------|-----------|
| `NULL` (default) | Orphan notes only — notes with `project_id IS NULL` |
| `-1` | All notes — opt-in to global search |
| Positive int | That project's notes only |
`build_context()` derives `orphan_only` + `effective_project_id` from this value before calling both the semantic and keyword search paths.
**Scope tools** — Two LLM tools let the model discover and switch scope mid-conversation:
- `search_projects` — SequenceMatcher scoring over title + description + `auto_summary`; returns top 5 matching projects.
- `set_rag_scope` — persists the new `rag_project_id` to DB immediately; blocked in workspace view; causes the SSE `done` event to include `new_rag_scope` + `new_rag_scope_label` so the frontend chip updates reactively.
**Project summaries**`generate_project_summary()` calls Ollama (fire-and-forget) and stores the result in `projects.auto_summary`. Triggered on project update and note saves (debounced 1h). `backfill_project_summaries()` runs at startup.
Embedding model: `nomic-embed-text` via Ollama. Backfill runs 30s after startup (background task). Embedding model: `nomic-embed-text` via Ollama. Backfill runs 30s after startup (background task).
+11 -4
View File
@@ -43,7 +43,7 @@ Tasks carry status (`todo` → `in_progress` → `done`), priority (`none`/`low`
Full conversation history with SSE streaming. Features: Full conversation history with SSE streaming. Features:
- **RAG** — Semantically relevant notes (≥ 0.60 cosine similarity) auto-injected as context. Notes 0.450.60 shown in sidebar as "Suggested." - **RAG** — Semantically relevant notes (≥ 0.60 cosine similarity) auto-injected as context. Notes 0.450.60 shown in sidebar as "Suggested."
- **Attach notes** — Paperclip icon to include specific notes in context. - **Attach notes** — Paperclip icon to include specific notes in context.
- **Project scope** — Restrict RAG to a single project's notes via the sidebar selector. - **RAG scope chip** — Pill above the input bar shows the current note scope. Click to switch: "Orphan notes only" (default — project notes stay out of general chat), any active project, or "All notes." Scope is persisted per conversation. The AI can also call `search_projects` and `set_rag_scope` mid-conversation to switch scope automatically; the chip pulses when this happens.
- **Tool calls** — The assistant can create/update notes, tasks, projects, milestones, search the web, check weather, read RSS, query calendar events, and more. Tool calls display inline with confirm/deny for creates. - **Tool calls** — The assistant can create/update notes, tasks, projects, milestones, search the web, check weather, read RSS, query calendar events, and more. Tool calls display inline with confirm/deny for creates.
- **Thinking mode** — Toggle extended reasoning for complex questions. - **Thinking mode** — Toggle extended reasoning for complex questions.
- **Abort** — Stop button cancels in-flight generation. - **Abort** — Stop button cancels in-flight generation.
@@ -70,9 +70,17 @@ Full conversation history with SSE streaming. Features:
The assistant can search the web (SearXNG) and fetch pages, synthesising findings into notes. A lightweight `search_web` tool answers quick questions inline without saving. Requires `SEARXNG_URL` to be configured. The assistant can search the web (SearXNG) and fetch pages, synthesising findings into notes. A lightweight `search_web` tool answers quick questions inline without saving. Requires `SEARXNG_URL` to be configured.
## CalDAV Calendar ## Calendar
Connect an external CalDAV server (Nextcloud, Radicale, etc.) and manage calendar events via natural language. Tools: `create_event`, `list_events`, `search_events`, `update_event`, `delete_event`. Activated when user configures CalDAV credentials in Settings → Integrations. `/calendar` shows a full FullCalendar view (month, week, day). Click an empty slot to create an event; click an existing event to edit or delete it via a slide-over panel.
**Internal events store** — Events are stored in the app database (`events` table), making them available without any external calendar. Fields: title, description, start/end datetime, all-day toggle, location, colour.
**AI tools**`create_event`, `list_events`, `search_events`, `update_event`, `delete_event` all operate on the internal store. Tool-call result cards in chat are clickable and open the same EventSlideOver for editing.
**HomeView widget** — The dashboard shows today's and the next 7 days' events as clickable cards above the hero project.
**CalDAV sync (optional)** — Connect an external CalDAV server (Nextcloud, Radicale, etc.) in Settings → Integrations. Events sync bidirectionally via a `caldav_uid` field.
## Sharing and Collaboration ## Sharing and Collaboration
@@ -119,7 +127,6 @@ Settings are tabbed:
## Roadmap ## Roadmap
- Calendar/timeline view for tasks with due dates
- Email integration (read/send via IMAP/SMTP tools in chat) - Email integration (read/send via IMAP/SMTP tools in chat)
- Session invalidation on user deletion - Session invalidation on user deletion
- Flutter push notifications (requires FCM/APNs — separate from web VAPID) - Flutter push notifications (requires FCM/APNs — separate from web VAPID)