FabledScribe/docs/specs/2026-03-25-rag-scoping-design.md

RAG Scoping and Context Isolation Design

For agentic workers: Use superpowers:executing-plans or superpowers:subagent-driven-development to implement this plan task-by-task.

Goal: Prevent project-associated notes from polluting global chat RAG, and give the LLM tools to discover and switch project scope during a conversation.

Problem being solved: Users with multiple projects (e.g. scifi worldbuilding, personal productivity) find that semantic search pulls notes from whichever project has the most content, contaminating general-purpose chat context. Project notes should be siloed by default.

---

Architecture

Default RAG behavior change

Currently rag_project_id=None in build_context() searches all notes. After this change the semantics become a three-value system:

• rag_project_id = None → search only orphan notes (project_id IS NULL)
• rag_project_id = <positive int> → search only that project's notes
• rag_project_id = -1 → search all notes (explicit opt-in to current global behavior)

semantic_search_notes() change (services/embeddings.py): add orphan_only: bool = False parameter. When True, add .where(Note.project_id.is_(None)) to the SQLAlchemy query.

search_notes_for_context() change (services/notes.py): add orphan_only: bool = False parameter with the same filter.

build_context() change (services/llm.py): compute the flags before calling search:

# rag_project_id=None → orphan only; -1 → all notes; positive int → that project
orphan_only = rag_project_id is None
effective_project_id = rag_project_id if (rag_project_id is not None and rag_project_id != -1) else None

for score, note in await semantic_search_notes(
    user_id, user_message, exclude_ids=search_exclude or None, limit=8,
    project_id=effective_project_id,
    orphan_only=orphan_only,
):
    ...

The same orphan_only / effective_project_id pattern applies to the keyword fallback call to search_notes_for_context().

Conversation scope persistence

New DB column: conversations.rag_project_id INTEGER (nullable, default NULL).

Semantics match the three-value system above. NULL is the default for all new and existing conversations after migration — existing conversations transparently gain orphan-only scoping.

Model change (models/conversation.py):

rag_project_id: Mapped[int | None] = mapped_column(Integer, nullable=True, default=None)

to_dict() must include "rag_project_id": self.rag_project_id.

Route change (routes/chat.py): PATCH /api/chat/conversations/:id must accept rag_project_id in the request body and call update_conversation(conv_id, user_id, rag_project_id=value). GET /api/chat/conversations/:id returns rag_project_id via to_dict().

Service change (services/chat.py): update_conversation() must accept and persist rag_project_id.

Project summarization background job

New DB columns on projects:

auto_summary: Mapped[str | None] = mapped_column(Text, nullable=True)
summary_updated_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True)

generate_project_summary(user_id: int, project_id: int) -> None (services/projects.py):

• Fetches project metadata (title, description, goal)
• Fetches up to 10 notes for that project: SELECT title, body[:200] FROM notes WHERE project_id=? ORDER BY updated_at DESC LIMIT 10
• Builds an Ollama prompt: "Summarize this project in 3-4 sentences covering its purpose, themes, and content. Title: {title}. Description: {description}. Goal: {goal}. Recent notes: {sample}"
• Calls Ollama (same generate_completion() helper used elsewhere, stream=False)
• Stores result in project.auto_summary and project.summary_updated_at = datetime.now(UTC)
• On Ollama failure: log debug message, return without updating (graceful degradation)

backfill_project_summaries() -> None (services/projects.py): fire-and-forget, called at startup alongside backfill_note_embeddings(). Generates summaries for all projects where auto_summary IS NULL.

Trigger from update_project(): after committing, call asyncio.create_task(generate_project_summary(user_id, project_id)) unconditionally.

Trigger from note saves: in create_note() and update_note() within services/notes.py, when project_id is not None, check project.summary_updated_at. If None or more than 1 hour old, fire asyncio.create_task(generate_project_summary(user_id, project_id)). This avoids flooding Ollama during heavy note-editing sessions.

generate_project_summary and backfill_project_summaries are internal background tasks only, not exposed as LLM tools.

New LLM tools

Both tools are added to _CORE_TOOLS in services/tools.py.

search_projects(query: str):

• Loads all projects for the user from the DB (title, description, goal, auto_summary)
• Scores each project: SequenceMatcher(None, query.lower(), combined_text.lower()).ratio() where combined_text = f"{title} {description or ''} {auto_summary or ''}", plus keyword overlap bonus
• Returns top 5 as [{id, title, summary_snippet, score}] where summary_snippet is auto_summary[:200] or description[:200]
• Tool result type: "projects_list"

set_rag_scope(project_id: int | None):

• project_id = <positive int> → scope to that project
• project_id = null → orphan-only
• project_id = -1 → all notes
• Workspace guard: if workspace_project_id is set on the current generation context, return {success: false, error: "Cannot change RAG scope in workspace view"} and make no changes
• Validates positive project_id exists and belongs to user (returns error if not)
• Calls await update_conversation(conv_id, user_id, rag_project_id=project_id) to persist immediately
• Returns {success: true, type: "rag_scope_set", scope_label: "<human-readable>"} e.g. "Orphan notes only", "SciFi Project", "All notes"

conv_id threading (services/tools.py, services/generation_task.py):

Current execute_tool() signature:

async def execute_tool(user_id: int, tool_name: str, arguments: dict) -> dict

Updated signature:

async def execute_tool(
    user_id: int,
    tool_name: str,
    arguments: dict,
    conv_id: int | None = None,
    workspace_project_id: int | None = None,
) -> dict

In generation_task.py, the call site in the tool execution loop must pass conv_id=conv_id, workspace_project_id=workspace_project_id — both are already available as local variables in run_generation().

SSE new_rag_scope wiring:

In generation_task.py, after each tool call returns, check if result.get("type") == "rag_scope_set" and result.get("success"). Store new_rag_scope = arguments["project_id"] and new_rag_scope_label = result["scope_label"].

When emitting the final SSE done event (currently {"done": True, "message_id": msg_id, "timing": timing}), add the scope fields when present:

done_payload = {"done": True, "message_id": msg_id, "timing": timing}
if new_rag_scope_label is not None:
    done_payload["new_rag_scope"] = new_rag_scope        # raw int or None
    done_payload["new_rag_scope_label"] = new_rag_scope_label
yield f"data: {json.dumps(done_payload)}\n\n"

In stores/chat.ts, the SSE done handler must check event.data.new_rag_scope !== undefined and set currentConversation.value.rag_project_id = event.data.new_rag_scope.

Frontend scope indicator

Chat store (stores/chat.ts):

• currentConversation already holds the full conversation object; add rag_project_id: number | null to the conversation interface
• Add a computed ragProjectId that reads currentConversation.value?.rag_project_id ?? null
• Add async updateRagScope(convId: number, ragProjectId: number | null) that calls PATCH /api/chat/conversations/:id { rag_project_id } and updates currentConversation.value.rag_project_id locally on success
• In the SSE done handler, if event.data.new_rag_scope !== undefined, set currentConversation.value.rag_project_id = event.data.new_rag_scope

ChatView (views/ChatView.vue):

• Remove the existing ProjectSelector from the sidebar RAG scope section
• Add a scope chip just above the message input. Shows: ⊙ Orphan notes (null) / ⊙ All notes (-1) / ⊙ {project title} (positive int)
• Clicking opens a compact dropdown: "Orphan notes only", all active projects by title, "All notes"
• On selection, call chatStore.updateRagScope(convId, selectedValue)
• When chatStore.ragProjectId changes (from SSE or UI), briefly apply a CSS pulse animation to the chip
• Load the projects list once on mount via apiGet("/api/projects?status=active")

WorkspaceView — no change. It hardcodes ragProjectId to the workspace project's id (positive int). Scope chip is not shown in workspace view.

---

Files Changed

Backend

File	Change
alembic/versions/0030_rag_scoping.py	Add conversations.rag_project_id, projects.auto_summary, projects.summary_updated_at
models/conversation.py	Add rag_project_id: Mapped[int | None]; include in to_dict()
models/project.py	Add auto_summary: Mapped[str | None], summary_updated_at: Mapped[datetime | None]
services/projects.py	Add generate_project_summary(), backfill_project_summaries(); trigger from update_project()
services/notes.py	Add orphan_only param to search_notes_for_context(); trigger summary regen from note saves
services/embeddings.py	Add orphan_only: bool = False param to semantic_search_notes()
services/llm.py	Compute orphan_only + effective_project_id from rag_project_id; pass to both search calls
services/tools.py	Add search_projects and set_rag_scope definitions and handlers; update execute_tool() signature
services/generation_task.py	Pass conv_id + workspace_project_id to execute_tool(); capture scope change; emit in SSE done
services/chat.py	Add rag_project_id parameter to update_conversation()
routes/chat.py	Accept rag_project_id in PATCH body; return in GET responses via to_dict()
app.py	Call backfill_project_summaries() at startup

Frontend

File	Change
api/client.ts	Conversation type includes rag_project_id; helpers accept it
stores/chat.ts	ragProjectId computed; updateRagScope() method; SSE done handler updates scope
views/ChatView.vue	Replace sidebar ProjectSelector with scope chip above input bar

---

Data Flow

User changes scope via chip
  → chatStore.updateRagScope(convId, value)
  → PATCH /api/chat/conversations/:id { rag_project_id: value }
  → DB updated; currentConversation.rag_project_id updated locally
  → chip re-renders immediately

User sends message
  → sendMessage() reads chatStore.ragProjectId → passes as rag_project_id in POST body
  → build_context() derives orphan_only + effective_project_id
  → RAG results scoped accordingly

Model calls set_rag_scope(3) during generation
  → execute_tool(conv_id=X, workspace_project_id=None) writes rag_project_id=3 to DB
  → returns { success: true, type: "rag_scope_set", scope_label: "SciFi Project" }
  → generation_task stores new_rag_scope=3, new_rag_scope_label="SciFi Project"
  → SSE done event: { done: true, new_rag_scope: 3, new_rag_scope_label: "SciFi Project", ... }
  → chat store done handler: currentConversation.rag_project_id = 3
  → chip pulses → "⊙ SciFi Project"

User opens existing conversation
  → GET /api/chat/conversations/:id returns rag_project_id
  → chat store sets currentConversation; ragProjectId computed picks it up
  → chip renders correct scope immediately

---

Error Handling

• Ollama unavailable during summary gen: log debug, return without updating; old summary (or null) retained; search_projects falls back to title + description matching
• set_rag_scope called in workspace: return {success: false, error: "Cannot change RAG scope in workspace view"}; no DB write
• set_rag_scope with unknown positive project_id: return {success: false, error: "Project not found"}; scope unchanged
• search_projects with no summaries yet: score on title + description only; still returns results
• Existing conversations after migration: rag_project_id defaults to NULL → orphan-only; behavior silently improves without user action

---

Testing

• Unit: semantic_search_notes(orphan_only=True) returns only notes where project_id IS NULL
• Unit: semantic_search_notes(project_id=3, orphan_only=False) returns only notes for project 3
• Unit: build_context() passes correct orphan_only / effective_project_id for all three rag_project_id values (None, -1, positive)
• Unit: search_projects() scoring ranks correct project first for representative queries; degrades gracefully with no summaries
• Unit: generate_project_summary() builds correct Ollama prompt; handles Ollama failure gracefully
• Integration: set_rag_scope() handler writes to DB, returns correct scope_label, rejects in workspace context
• Integration: SSE done event includes new_rag_scope when scope changes mid-conversation
• Frontend: scope chip reflects loaded conversation scope; updates reactively on model tool call