From 1e0d11c9071cc789dd94b349c833d9135703b6e2 Mon Sep 17 00:00:00 2001 From: Bryan Van Deusen Date: Wed, 25 Mar 2026 17:01:30 -0400 Subject: [PATCH] docs: update RAG scoping spec with explicit wiring details Add exact code snippets for orphan_only logic, conv_id threading, SSE new_rag_scope wiring, workspace guard, and frontend store changes. Co-Authored-By: Claude Sonnet 4.6 --- docs/specs/2026-03-25-rag-scoping-design.md | 214 ++++++++++++++------ 1 file changed, 151 insertions(+), 63 deletions(-) diff --git a/docs/specs/2026-03-25-rag-scoping-design.md b/docs/specs/2026-03-25-rag-scoping-design.md index 58b7fde..d3f8a71 100644 --- a/docs/specs/2026-03-25-rag-scoping-design.md +++ b/docs/specs/2026-03-25-rag-scoping-design.md @@ -12,68 +12,146 @@ ### Default RAG behavior change -Currently `rag_project_id=None` in `build_context()` searches all notes. After this change: +Currently `rag_project_id=None` in `build_context()` searches all notes. After this change the semantics become a three-value system: - `rag_project_id = None` → search only orphan notes (`project_id IS NULL`) -- `rag_project_id = ` → search only that project's notes (existing behavior) +- `rag_project_id = ` → search only that project's notes - `rag_project_id = -1` → search all notes (explicit opt-in to current global behavior) -This change is isolated to `semantic_search_notes()` and `search_notes_for_context()` in `services/embeddings.py` and `services/notes.py`. A new `orphan_only: bool` parameter threads through from `build_context()`. +**`semantic_search_notes()` change** (`services/embeddings.py`): add `orphan_only: bool = False` parameter. When `True`, add `.where(Note.project_id.is_(None))` to the SQLAlchemy query. + +**`search_notes_for_context()` change** (`services/notes.py`): add `orphan_only: bool = False` parameter with the same filter. + +**`build_context()` change** (`services/llm.py`): compute the flags before calling search: + +```python +# rag_project_id=None → orphan only; -1 → all notes; positive int → that project +orphan_only = rag_project_id is None +effective_project_id = rag_project_id if (rag_project_id is not None and rag_project_id != -1) else None + +for score, note in await semantic_search_notes( + user_id, user_message, exclude_ids=search_exclude or None, limit=8, + project_id=effective_project_id, + orphan_only=orphan_only, +): + ... +``` + +The same `orphan_only` / `effective_project_id` pattern applies to the keyword fallback call to `search_notes_for_context()`. ### Conversation scope persistence -`conversations.rag_project_id INTEGER` (nullable) stores the active scope per conversation: -- `NULL` → orphan-only (default for new conversations) -- positive int → scoped to a project -- `-1` → all notes +**New DB column:** `conversations.rag_project_id INTEGER` (nullable, default `NULL`). -Loaded when a conversation is opened; written when the user changes the scope via the UI or when the model calls `set_rag_scope`. +Semantics match the three-value system above. `NULL` is the default for all new and existing conversations after migration — existing conversations transparently gain orphan-only scoping. + +**Model change** (`models/conversation.py`): +```python +rag_project_id: Mapped[int | None] = mapped_column(Integer, nullable=True, default=None) +``` + +`to_dict()` must include `"rag_project_id": self.rag_project_id`. + +**Route change** (`routes/chat.py`): `PATCH /api/chat/conversations/:id` must accept `rag_project_id` in the request body and call `update_conversation(conv_id, user_id, rag_project_id=value)`. `GET /api/chat/conversations/:id` returns `rag_project_id` via `to_dict()`. + +**Service change** (`services/chat.py`): `update_conversation()` must accept and persist `rag_project_id`. ### Project summarization background job -Each project gets an `auto_summary TEXT` and `summary_updated_at TIMESTAMP` column. A new `generate_project_summary(user_id, project_id)` function calls Ollama with: -- Project title, description, goal -- Up to 10 note titles + first 200 chars of their body +**New DB columns** on `projects`: +```python +auto_summary: Mapped[str | None] = mapped_column(Text, nullable=True) +summary_updated_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True) +``` -Triggers: -1. **Startup backfill** — any project with `auto_summary IS NULL` gets generated (fire-and-forget, alongside `backfill_note_embeddings`) -2. **On project update** — `update_project()` fires a background task unconditionally -3. **On note save to a project** — `create_note` / `update_note` with `project_id` checks if `summary_updated_at` is more than 1 hour old before triggering (debounce) +**`generate_project_summary(user_id: int, project_id: int) -> None`** (`services/projects.py`): +- Fetches project metadata (title, description, goal) +- Fetches up to 10 notes for that project: `SELECT title, body[:200] FROM notes WHERE project_id=? ORDER BY updated_at DESC LIMIT 10` +- Builds an Ollama prompt: "Summarize this project in 3-4 sentences covering its purpose, themes, and content. Title: {title}. Description: {description}. Goal: {goal}. Recent notes: {sample}" +- Calls Ollama (same `generate_completion()` helper used elsewhere, `stream=False`) +- Stores result in `project.auto_summary` and `project.summary_updated_at = datetime.now(UTC)` +- On Ollama failure: log debug message, return without updating (graceful degradation) -Graceful degradation: if Ollama is unavailable the function exits silently; the old summary (or `None`) is retained. The project search tool falls back to title + description matching only. +**`backfill_project_summaries() -> None`** (`services/projects.py`): fire-and-forget, called at startup alongside `backfill_note_embeddings()`. Generates summaries for all projects where `auto_summary IS NULL`. + +**Trigger from `update_project()`**: after committing, call `asyncio.create_task(generate_project_summary(user_id, project_id))` unconditionally. + +**Trigger from note saves**: in `create_note()` and `update_note()` within `services/notes.py`, when `project_id` is not None, check `project.summary_updated_at`. If `None` or more than 1 hour old, fire `asyncio.create_task(generate_project_summary(user_id, project_id))`. This avoids flooding Ollama during heavy note-editing sessions. + +`generate_project_summary` and `backfill_project_summaries` are **internal background tasks only**, not exposed as LLM tools. ### New LLM tools -Both tools are added to `_CORE_TOOLS` (always available, no integration dependency). +Both tools are added to `_CORE_TOOLS` in `services/tools.py`. -**`search_projects(query: str)`** -- Loads all active projects for the user from the DB -- Scores each against the query: SequenceMatcher ratio on title + description + auto_summary combined, plus keyword overlap -- Returns top 5 as `[{id, title, summary_snippet, score}]` -- Used by the model to identify which project the user is referring to +**`search_projects(query: str)`**: +- Loads all projects for the user from the DB (title, description, goal, auto_summary) +- Scores each project: `SequenceMatcher(None, query.lower(), combined_text.lower()).ratio()` where `combined_text = f"{title} {description or ''} {auto_summary or ''}"`, plus keyword overlap bonus +- Returns top 5 as `[{id, title, summary_snippet, score}]` where `summary_snippet` is `auto_summary[:200]` or `description[:200]` +- Tool result type: `"projects_list"` -**`set_rag_scope(project_id: int | None)`** +**`set_rag_scope(project_id: int | None)`**: - `project_id = ` → scope to that project - `project_id = null` → orphan-only - `project_id = -1` → all notes -- Side effects: (1) writes `conversations.rag_project_id` immediately, (2) returns `{success, scope_label}` for model to confirm to user, (3) emits `new_rag_scope` in the SSE `done` event metadata so the frontend scope chip updates reactively +- **Workspace guard:** if `workspace_project_id` is set on the current generation context, return `{success: false, error: "Cannot change RAG scope in workspace view"}` and make no changes +- Validates positive project_id exists and belongs to user (returns error if not) +- Calls `await update_conversation(conv_id, user_id, rag_project_id=project_id)` to persist immediately +- Returns `{success: true, type: "rag_scope_set", scope_label: ""}` e.g. `"Orphan notes only"`, `"SciFi Project"`, `"All notes"` -`set_rag_scope` needs `conv_id` available in the tool executor — it is already threaded through `generation_task.py` → `execute_tool()`. +**`conv_id` threading** (`services/tools.py`, `services/generation_task.py`): -Typical model flow: -> User: "let's talk about my scifi project" -> Model calls `search_projects("scifi")` → finds project id 3 → calls `set_rag_scope(3)` → responds "I've scoped our conversation to your SciFi Project." +Current `execute_tool()` signature: +```python +async def execute_tool(user_id: int, tool_name: str, arguments: dict) -> dict +``` + +Updated signature: +```python +async def execute_tool( + user_id: int, + tool_name: str, + arguments: dict, + conv_id: int | None = None, + workspace_project_id: int | None = None, +) -> dict +``` + +In `generation_task.py`, the call site in the tool execution loop must pass `conv_id=conv_id, workspace_project_id=workspace_project_id` — both are already available as local variables in `run_generation()`. + +**SSE `new_rag_scope` wiring:** + +In `generation_task.py`, after each tool call returns, check `if result.get("type") == "rag_scope_set" and result.get("success")`. Store `new_rag_scope = arguments["project_id"]` and `new_rag_scope_label = result["scope_label"]`. + +When emitting the final SSE `done` event (currently `{"done": True, "message_id": msg_id, "timing": timing}`), add the scope fields when present: + +```python +done_payload = {"done": True, "message_id": msg_id, "timing": timing} +if new_rag_scope_label is not None: + done_payload["new_rag_scope"] = new_rag_scope # raw int or None + done_payload["new_rag_scope_label"] = new_rag_scope_label +yield f"data: {json.dumps(done_payload)}\n\n" +``` + +In `stores/chat.ts`, the SSE `done` handler must check `event.data.new_rag_scope !== undefined` and set `currentConversation.value.rag_project_id = event.data.new_rag_scope`. ### Frontend scope indicator -A scope chip sits just above the message input bar (replaces the sidebar `ProjectSelector`). It shows the active scope: `⊙ Orphan notes` / `⊙ SciFi Project` / `⊙ All notes`. Clicking opens a compact dropdown listing: -- "Orphan notes only" (default) -- All active projects by title -- "All notes" +**Chat store** (`stores/chat.ts`): +- `currentConversation` already holds the full conversation object; add `rag_project_id: number | null` to the conversation interface +- Add a computed `ragProjectId` that reads `currentConversation.value?.rag_project_id ?? null` +- Add `async updateRagScope(convId: number, ragProjectId: number | null)` that calls `PATCH /api/chat/conversations/:id { rag_project_id }` and updates `currentConversation.value.rag_project_id` locally on success +- In the SSE `done` handler, if `event.data.new_rag_scope !== undefined`, set `currentConversation.value.rag_project_id = event.data.new_rag_scope` -Selecting an option calls `PATCH /api/chat/conversations/:id` with `{ rag_project_id }` and updates local state immediately. +**ChatView** (`views/ChatView.vue`): +- Remove the existing `ProjectSelector` from the sidebar RAG scope section +- Add a scope chip just above the message input. Shows: `⊙ Orphan notes` (null) / `⊙ All notes` (-1) / `⊙ {project title}` (positive int) +- Clicking opens a compact dropdown: "Orphan notes only", all active projects by title, "All notes" +- On selection, call `chatStore.updateRagScope(convId, selectedValue)` +- When `chatStore.ragProjectId` changes (from SSE or UI), briefly apply a CSS pulse animation to the chip +- Load the projects list once on mount via `apiGet("/api/projects?status=active")` -When the model calls `set_rag_scope`, the SSE `done` event's `new_rag_scope` field causes the chat store to update `ragProjectId` for the active conversation, re-rendering the chip with a brief pulse animation. +**WorkspaceView** — no change. It hardcodes `ragProjectId` to the workspace project's id (positive int). Scope chip is not shown in workspace view. --- @@ -84,25 +162,25 @@ When the model calls `set_rag_scope`, the SSE `done` event's `new_rag_scope` fie | File | Change | |------|--------| | `alembic/versions/0030_rag_scoping.py` | Add `conversations.rag_project_id`, `projects.auto_summary`, `projects.summary_updated_at` | -| `models/conversation.py` | Add `rag_project_id: Mapped[int \| None]` | +| `models/conversation.py` | Add `rag_project_id: Mapped[int \| None]`; include in `to_dict()` | | `models/project.py` | Add `auto_summary: Mapped[str \| None]`, `summary_updated_at: Mapped[datetime \| None]` | | `services/projects.py` | Add `generate_project_summary()`, `backfill_project_summaries()`; trigger from `update_project()` | -| `services/notes.py` | Trigger summary regeneration (debounced) from `create_note()` / `update_note()` when `project_id` is set | +| `services/notes.py` | Add `orphan_only` param to `search_notes_for_context()`; trigger summary regen from note saves | | `services/embeddings.py` | Add `orphan_only: bool = False` param to `semantic_search_notes()` | -| `services/notes.py` | Add `orphan_only` param to `search_notes_for_context()` | -| `services/llm.py` | Pass `orphan_only=True` when `rag_project_id` is `None`; pass `orphan_only=False` when `-1`; add `new_rag_scope` to SSE done metadata | -| `services/tools.py` | Add `search_projects` and `set_rag_scope` tool definitions and handlers | -| `services/generation_task.py` | Thread `conv_id` into `execute_tool()`; apply `new_rag_scope` from tool result to context meta | -| `routes/chat.py` | Include `rag_project_id` in conversation responses; handle `PATCH` body field | +| `services/llm.py` | Compute `orphan_only` + `effective_project_id` from `rag_project_id`; pass to both search calls | +| `services/tools.py` | Add `search_projects` and `set_rag_scope` definitions and handlers; update `execute_tool()` signature | +| `services/generation_task.py` | Pass `conv_id` + `workspace_project_id` to `execute_tool()`; capture scope change; emit in SSE done | +| `services/chat.py` | Add `rag_project_id` parameter to `update_conversation()` | +| `routes/chat.py` | Accept `rag_project_id` in PATCH body; return in GET responses via `to_dict()` | | `app.py` | Call `backfill_project_summaries()` at startup | ### Frontend | File | Change | |------|--------| -| `api/client.ts` | Add `updateConversationScope(convId, ragProjectId)` helper | -| `stores/chat.ts` | Load `ragProjectId` per conversation; update on `new_rag_scope` SSE metadata; expose setter | -| `views/ChatView.vue` | Replace sidebar `ProjectSelector` with scope chip above input bar; load projects list for dropdown | +| `api/client.ts` | Conversation type includes `rag_project_id`; helpers accept it | +| `stores/chat.ts` | `ragProjectId` computed; `updateRagScope()` method; SSE done handler updates scope | +| `views/ChatView.vue` | Replace sidebar `ProjectSelector` with scope chip above input bar | --- @@ -110,39 +188,49 @@ When the model calls `set_rag_scope`, the SSE `done` event's `new_rag_scope` fie ``` User changes scope via chip - → PATCH /api/chat/conversations/:id { rag_project_id } - → conversations.rag_project_id updated + → chatStore.updateRagScope(convId, value) + → PATCH /api/chat/conversations/:id { rag_project_id: value } + → DB updated; currentConversation.rag_project_id updated locally → chip re-renders immediately User sends message - → chat store sends rag_project_id with message - → build_context() applies orphan/project/all filter + → sendMessage() reads chatStore.ragProjectId → passes as rag_project_id in POST body + → build_context() derives orphan_only + effective_project_id → RAG results scoped accordingly -Model calls set_rag_scope(3) - → execute_tool() writes conversations.rag_project_id = 3 - → returns { success, scope_label: "SciFi Project" } - → generation_task adds new_rag_scope to done metadata - → chat store receives new_rag_scope → updates ragProjectId - → chip pulses and shows "⊙ SciFi Project" +Model calls set_rag_scope(3) during generation + → execute_tool(conv_id=X, workspace_project_id=None) writes rag_project_id=3 to DB + → returns { success: true, type: "rag_scope_set", scope_label: "SciFi Project" } + → generation_task stores new_rag_scope=3, new_rag_scope_label="SciFi Project" + → SSE done event: { done: true, new_rag_scope: 3, new_rag_scope_label: "SciFi Project", ... } + → chat store done handler: currentConversation.rag_project_id = 3 + → chip pulses → "⊙ SciFi Project" + +User opens existing conversation + → GET /api/chat/conversations/:id returns rag_project_id + → chat store sets currentConversation; ragProjectId computed picks it up + → chip renders correct scope immediately ``` --- ## Error Handling -- **Ollama unavailable during summary gen:** silent skip, retain existing summary -- **set_rag_scope with unknown project_id:** return `{success: false, error: "Project not found"}`; scope unchanged -- **search_projects with no summaries yet:** fall back to title + description matching only; still returns results -- **Conversation has no rag_project_id column yet (pre-migration):** default to `None` (orphan-only); non-breaking +- **Ollama unavailable during summary gen:** log debug, return without updating; old summary (or null) retained; `search_projects` falls back to title + description matching +- **`set_rag_scope` called in workspace:** return `{success: false, error: "Cannot change RAG scope in workspace view"}`; no DB write +- **`set_rag_scope` with unknown positive project_id:** return `{success: false, error: "Project not found"}`; scope unchanged +- **`search_projects` with no summaries yet:** score on title + description only; still returns results +- **Existing conversations after migration:** `rag_project_id` defaults to `NULL` → orphan-only; behavior silently improves without user action --- ## Testing -- Unit: `semantic_search_notes()` with `orphan_only=True` excludes project notes -- Unit: `search_projects()` scoring ranks correct project first for representative queries -- Unit: `generate_project_summary()` builds correct prompt and stores result -- Integration: `set_rag_scope()` tool handler writes to DB and returns correct scope_label +- Unit: `semantic_search_notes(orphan_only=True)` returns only notes where `project_id IS NULL` +- Unit: `semantic_search_notes(project_id=3, orphan_only=False)` returns only notes for project 3 +- Unit: `build_context()` passes correct `orphan_only` / `effective_project_id` for all three `rag_project_id` values (None, -1, positive) +- Unit: `search_projects()` scoring ranks correct project first for representative queries; degrades gracefully with no summaries +- Unit: `generate_project_summary()` builds correct Ollama prompt; handles Ollama failure gracefully +- Integration: `set_rag_scope()` handler writes to DB, returns correct scope_label, rejects in workspace context - Integration: SSE `done` event includes `new_rag_scope` when scope changes mid-conversation -- Frontend: scope chip reflects loaded conversation scope; updates on model tool call +- Frontend: scope chip reflects loaded conversation scope; updates reactively on model tool call