FabledScribe/docs/plans/2026-03-25-rag-scoping.md

# RAG Scoping and Context Isolation Implementation Plan

> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.

**Goal:** Silo project notes from global chat RAG by default, persist scope per conversation, and give the LLM `search_projects` and `set_rag_scope` tools.

**Architecture:** Add `rag_project_id` to conversations (NULL=orphan-only, -1=all, positive=project). Add `auto_summary` to projects, generated by a background job. New `orphan_only` param threads from `build_context()` down to the search functions. Two new LLM tools let the model discover and switch scope.

**Tech Stack:** Python/Quart/SQLAlchemy (backend), Vue 3/TypeScript/Pinia (frontend), Ollama (summary generation), pytest + AsyncMock (tests)

**Spec:** `docs/specs/2026-03-25-rag-scoping-design.md`

---

### Task 1: Migration 0030 — rag_project_id + project summary columns

**Files:**
- Create: `alembic/versions/0030_rag_scoping.py`

- [ ] **Step 1: Create the migration file**

```python
# alembic/versions/0030_rag_scoping.py
"""Add rag_project_id to conversations; auto_summary columns to projects."""

from alembic import op

revision = "0030"
down_revision = "0029"


def upgrade() -> None:
    op.execute("""
        ALTER TABLE conversations
            ADD COLUMN IF NOT EXISTS rag_project_id INTEGER DEFAULT NULL
    """)
    op.execute("""
        ALTER TABLE projects
            ADD COLUMN IF NOT EXISTS auto_summary TEXT DEFAULT NULL,
            ADD COLUMN IF NOT EXISTS summary_updated_at TIMESTAMPTZ DEFAULT NULL
    """)


def downgrade() -> None:
    op.execute("ALTER TABLE conversations DROP COLUMN IF EXISTS rag_project_id")
    op.execute("ALTER TABLE projects DROP COLUMN IF EXISTS auto_summary, DROP COLUMN IF EXISTS summary_updated_at")
```

- [ ] **Step 2: Commit**

```bash
git add alembic/versions/0030_rag_scoping.py
git commit -m "feat(rag): migration 0030 — add rag_project_id and project summary columns"
```

---

### Task 2: Update models — Conversation and Project

**Files:**
- Modify: `src/fabledassistant/models/conversation.py`
- Modify: `src/fabledassistant/models/project.py`

- [ ] **Step 1: Add `rag_project_id` to Conversation model**

In `models/conversation.py`, after the `briefing_date` field (line 25), add:

```python
rag_project_id: Mapped[int | None] = mapped_column(Integer, nullable=True, default=None)
```

In `to_dict()`, add after `"briefing_date"`:

```python
"rag_project_id": self.rag_project_id,
```

- [ ] **Step 2: Add summary fields to Project model**

In `models/project.py`, add imports `DateTime` from sqlalchemy and `datetime` from datetime. After the `color` field add:

```python
auto_summary: Mapped[str | None] = mapped_column(Text, nullable=True, default=None)
summary_updated_at: Mapped[datetime | None] = mapped_column(
    DateTime(timezone=True), nullable=True, default=None
)
```

Full updated imports needed at top of file:
```python
import enum
from datetime import datetime
from sqlalchemy import DateTime, ForeignKey, Integer, Text
from sqlalchemy.orm import Mapped, mapped_column
from fabledassistant.models import Base
from fabledassistant.models.base import TimestampMixin
```

- [ ] **Step 3: Commit**

```bash
git add src/fabledassistant/models/conversation.py src/fabledassistant/models/project.py
git commit -m "feat(rag): add rag_project_id to Conversation; auto_summary to Project"
```

---

### Task 3: Update chat service and route for rag_project_id

**Files:**
- Modify: `src/fabledassistant/services/chat.py:142-165`
- Modify: `src/fabledassistant/routes/chat.py:115-127`

- [ ] **Step 1: Add `rag_project_id` param to `update_conversation()`**

In `services/chat.py`, update the `update_conversation()` signature and body:

```python
async def update_conversation(
    user_id: int,
    conversation_id: int,
    title: str | None = None,
    model: str | None = None,
    rag_project_id: int | None | bool = False,  # False = "not provided"
) -> Conversation | None:
    async with async_session() as session:
        result = await session.execute(
            select(Conversation).where(
                Conversation.id == conversation_id,
                Conversation.user_id == user_id,
            )
        )
        conv = result.scalars().first()
        if conv is None:
            return None
        if title is not None:
            conv.title = title
        if model is not None:
            conv.model = model
        if rag_project_id is not False:
            conv.rag_project_id = rag_project_id  # type: ignore[assignment]
        conv.updated_at = datetime.now(timezone.utc)
        await session.commit()
        await session.refresh(conv)
        return conv
```

Note: using `False` as sentinel (not `None`) because `None` is a valid value to set (orphan-only).

- [ ] **Step 2: Update PATCH route to accept `rag_project_id`**

In `routes/chat.py`, replace the `update_conversation_route` function:

```python
@chat_bp.route("/conversations/<int:conv_id>", methods=["PATCH"])
@login_required
async def update_conversation_route(conv_id: int):
    uid = get_current_user_id()
    data = await request.get_json()
    title = data.get("title")
    model = data.get("model")
    # rag_project_id: use sentinel _MISSING to distinguish null from not-provided
    _MISSING = object()
    rag_project_id = data.get("rag_project_id", _MISSING)
    if title is None and model is None and rag_project_id is _MISSING:
        return jsonify({"error": "title, model, or rag_project_id is required"}), 400
    kwargs: dict = {}
    if title is not None:
        kwargs["title"] = title
    if model is not None:
        kwargs["model"] = model
    if rag_project_id is not _MISSING:
        kwargs["rag_project_id"] = rag_project_id
    conv = await update_conversation(uid, conv_id, **kwargs)
    if conv is None:
        return not_found("Conversation")
    return jsonify(conv.to_dict())
```

- [ ] **Step 3: Commit**

```bash
git add src/fabledassistant/services/chat.py src/fabledassistant/routes/chat.py
git commit -m "feat(rag): expose rag_project_id on conversation PATCH + service"
```

---

### Task 4: Add `orphan_only` param to search functions

**Files:**
- Modify: `src/fabledassistant/services/embeddings.py:76-130`
- Modify: `src/fabledassistant/services/notes.py:300-323`
- Modify: `tests/test_rag_scoping.py` (create)

- [ ] **Step 1: Write tests**

Create `tests/test_rag_scoping.py`:

```python
"""Tests for RAG scoping: orphan_only filtering and build_context scope logic."""
import pytest
from unittest.mock import AsyncMock, MagicMock, patch


def _make_note(id, project_id=None):
    n = MagicMock()
    n.id = id
    n.project_id = project_id
    n.title = f"Note {id}"
    n.body = "body"
    n.status = None
    return n


@pytest.mark.asyncio
async def test_semantic_search_orphan_only_adds_filter():
    """semantic_search_notes(orphan_only=True) must add project_id IS NULL filter."""
    from fabledassistant.services.embeddings import semantic_search_notes

    with patch("fabledassistant.services.embeddings.get_embedding", new_callable=AsyncMock) as mock_embed, \
         patch("fabledassistant.services.embeddings.async_session") as mock_session_cls:

        mock_embed.return_value = [0.1] * 768

        mock_session = AsyncMock()
        mock_session.__aenter__ = AsyncMock(return_value=mock_session)
        mock_session.__aexit__ = AsyncMock(return_value=False)
        mock_session.execute = AsyncMock(return_value=MagicMock(all=MagicMock(return_value=[])))
        mock_session_cls.return_value = mock_session

        result = await semantic_search_notes(1, "query", orphan_only=True)
        assert result == []
        # The key assertion is that it ran without error — the filter is applied internally.
        # Full DB-level testing is done via integration tests.


@pytest.mark.asyncio
async def test_build_context_scope_logic():
    """build_context() derives correct orphan_only + effective_project_id."""
    from fabledassistant.services import llm as llm_mod

    calls = []

    async def fake_search(user_id, query, exclude_ids=None, limit=8,
                          project_id=None, orphan_only=False):
        calls.append({"project_id": project_id, "orphan_only": orphan_only})
        return []

    with patch.object(llm_mod, "semantic_search_notes", fake_search), \
         patch("fabledassistant.services.llm.get_setting", new_callable=AsyncMock, return_value="Fable"), \
         patch("fabledassistant.services.llm.is_caldav_configured", new_callable=AsyncMock, return_value=False), \
         patch("fabledassistant.services.llm.get_note", new_callable=AsyncMock, return_value=None):

        # None → orphan only
        await llm_mod.build_context(1, [], None, "hello", rag_project_id=None)
        assert calls[-1] == {"project_id": None, "orphan_only": True}

        # -1 → all notes
        await llm_mod.build_context(1, [], None, "hello", rag_project_id=-1)
        assert calls[-1] == {"project_id": None, "orphan_only": False}

        # 5 → project 5
        await llm_mod.build_context(1, [], None, "hello", rag_project_id=5)
        assert calls[-1] == {"project_id": 5, "orphan_only": False}
```

- [ ] **Step 2: Run tests — expect failures**

```bash
docker compose exec app pytest tests/test_rag_scoping.py -v 2>&1 | tail -20
```

Expected: FAIL (orphan_only param not yet on functions)

- [ ] **Step 3: Add `orphan_only` to `semantic_search_notes()`**

In `services/embeddings.py`, update the signature and query:

```python
async def semantic_search_notes(
    user_id: int,
    query: str,
    exclude_ids: set[int] | None = None,
    limit: int = 8,
    threshold: float = _SIMILARITY_THRESHOLD,
    project_id: int | None = None,
    is_task: bool | None = None,
    orphan_only: bool = False,
) -> list[tuple[float, Note]]:
```

After the `is_task` filter block and before `if exclude_ids:`, add:

```python
if orphan_only:
    stmt = stmt.where(Note.project_id.is_(None))
```

- [ ] **Step 4: Add `orphan_only` to `search_notes_for_context()`**

In `services/notes.py`, update the signature:

```python
async def search_notes_for_context(
    user_id: int,
    keywords: list[str],
    exclude_ids: set[int] | None = None,
    limit: int = 3,
    project_id: int | None = None,
    orphan_only: bool = False,
) -> list[Note]:
```

After the `project_id` filter block and before `if exclude_ids:`, add:

```python
if orphan_only:
    query = query.where(Note.project_id.is_(None))
```

- [ ] **Step 5: Run tests — expect pass**

```bash
docker compose exec app pytest tests/test_rag_scoping.py::test_semantic_search_orphan_only_adds_filter -v
```

Expected: PASS

- [ ] **Step 6: Commit**

```bash
git add src/fabledassistant/services/embeddings.py src/fabledassistant/services/notes.py tests/test_rag_scoping.py
git commit -m "feat(rag): add orphan_only param to semantic_search_notes and search_notes_for_context"
```

---

### Task 5: Update `build_context()` for three-value scope semantics

**Files:**
- Modify: `src/fabledassistant/services/llm.py` (lines ~539–560)

- [ ] **Step 1: Update the two search call sites in `build_context()`**

Locate the semantic search block (around line 539) and the keyword fallback (around line 554). Replace both calls to use the derived scope:

```python
# Compute scope flags from rag_project_id three-value system
orphan_only = rag_project_id is None
effective_project_id = rag_project_id if (rag_project_id is not None and rag_project_id != -1) else None

# Try semantic search first; fall back to keyword search on failure / no results.
try:
    from fabledassistant.services.embeddings import semantic_search_notes
    for score, note in await semantic_search_notes(
        user_id, user_message, exclude_ids=search_exclude or None, limit=8,
        project_id=effective_project_id,
        orphan_only=orphan_only,
    ):
        found_scored.append((score, note))
except Exception:
    logger.warning("Semantic note search failed, falling back to keyword search", exc_info=True)

if not found_scored:
    keywords = _extract_keywords(user_message)
    if keywords:
        try:
            for note in await search_notes_for_context(
                user_id, keywords, exclude_ids=search_exclude or None, limit=8,
                project_id=effective_project_id,
                orphan_only=orphan_only,
            ):
                found_scored.append((None, note))
        except Exception:
            logger.warning("Failed to search notes for context", exc_info=True)
```

- [ ] **Step 2: Run scope logic tests**

```bash
docker compose exec app pytest tests/test_rag_scoping.py -v
```

Expected: all PASS

- [ ] **Step 3: Commit**

```bash
git add src/fabledassistant/services/llm.py
git commit -m "feat(rag): build_context derives orphan_only from rag_project_id three-value semantics"
```

---

### Task 6: Project summarization background job

**Files:**
- Modify: `src/fabledassistant/services/projects.py`
- Modify: `src/fabledassistant/services/notes.py` (create_note / update_note triggers)
- Modify: `src/fabledassistant/app.py`
- Modify: `tests/test_rag_scoping.py`

- [ ] **Step 1: Write tests for generate_project_summary**

Append to `tests/test_rag_scoping.py`:

```python
@pytest.mark.asyncio
async def test_generate_project_summary_stores_result():
    """generate_project_summary() writes auto_summary to project."""
    from fabledassistant.services.projects import generate_project_summary

    mock_project = MagicMock()
    mock_project.id = 1
    mock_project.title = "SciFi World"
    mock_project.description = "A space opera"
    mock_project.goal = "Finish the novel"
    mock_project.auto_summary = None

    mock_note = MagicMock()
    mock_note.title = "Chapter 1"
    mock_note.body = "It was a dark night..."

    mock_session = AsyncMock()
    mock_session.__aenter__ = AsyncMock(return_value=mock_session)
    mock_session.__aexit__ = AsyncMock(return_value=False)
    mock_session.commit = AsyncMock()
    mock_session.refresh = AsyncMock()
    execute_result = MagicMock()
    execute_result.scalars.return_value.first.return_value = mock_project
    execute_result.scalars.return_value.all.return_value = [mock_note]
    mock_session.execute = AsyncMock(return_value=execute_result)

    with patch("fabledassistant.services.projects.async_session", return_value=mock_session), \
         patch("fabledassistant.services.projects.generate_completion",
               new_callable=AsyncMock, return_value="A space opera about finishing a novel."):
        await generate_project_summary(1, 1)
        assert mock_project.auto_summary == "A space opera about finishing a novel."


@pytest.mark.asyncio
async def test_generate_project_summary_handles_ollama_failure():
    """generate_project_summary() swallows Ollama errors gracefully."""
    from fabledassistant.services.projects import generate_project_summary

    mock_project = MagicMock()
    mock_project.title = "Test"
    mock_project.description = ""
    mock_project.goal = ""
    mock_project.auto_summary = None

    mock_session = AsyncMock()
    mock_session.__aenter__ = AsyncMock(return_value=mock_session)
    mock_session.__aexit__ = AsyncMock(return_value=False)
    execute_result = MagicMock()
    execute_result.scalars.return_value.first.return_value = mock_project
    execute_result.scalars.return_value.all.return_value = []
    mock_session.execute = AsyncMock(return_value=execute_result)

    with patch("fabledassistant.services.projects.async_session", return_value=mock_session), \
         patch("fabledassistant.services.projects.generate_completion",
               side_effect=Exception("Ollama down")):
        # Must not raise
        await generate_project_summary(1, 1)
        assert mock_project.auto_summary is None
```

- [ ] **Step 2: Run tests — expect failures**

```bash
docker compose exec app pytest tests/test_rag_scoping.py::test_generate_project_summary_stores_result -v
```

Expected: FAIL (`generate_project_summary` not defined)

- [ ] **Step 3: Add `generate_project_summary` and `backfill_project_summaries` to `services/projects.py`**

Append to the end of `services/projects.py`:

```python
async def generate_project_summary(user_id: int, project_id: int) -> None:
    """Generate and store a concise project summary via Ollama. Fire-and-forget safe."""
    try:
        async with async_session() as session:
            project_result = await session.execute(
                select(Project).where(Project.id == project_id, Project.user_id == user_id)
            )
            project = project_result.scalars().first()
            if project is None:
                return

            notes_result = await session.execute(
                select(Note.title, Note.body)
                .where(Note.project_id == project_id, Note.user_id == user_id)
                .order_by(Note.updated_at.desc())
                .limit(10)
            )
            note_rows = notes_result.all()

        note_lines = "\n".join(
            f"- {row.title}: {(row.body or '')[:200]}" for row in note_rows
        )
        prompt = (
            f"Summarize this project in 3-4 sentences covering its purpose, themes, and content.\n"
            f"Title: {project.title}\n"
            f"Description: {project.description or '(none)'}\n"
            f"Goal: {project.goal or '(none)'}\n"
            f"Recent notes:\n{note_lines or '(none)'}"
        )

        from fabledassistant.services.llm import generate_completion
        summary = await generate_completion(prompt)
        if not summary:
            return

        async with async_session() as session:
            project_result = await session.execute(
                select(Project).where(Project.id == project_id, Project.user_id == user_id)
            )
            project = project_result.scalars().first()
            if project is None:
                return
            project.auto_summary = summary.strip()
            project.summary_updated_at = datetime.now(timezone.utc)
            await session.commit()

        logger.info("Generated summary for project %d", project_id)
    except Exception:
        logger.debug("Failed to generate summary for project %d", project_id, exc_info=True)


async def backfill_project_summaries() -> None:
    """Generate summaries for all projects that don't have one yet. Called at startup."""
    try:
        async with async_session() as session:
            result = await session.execute(
                select(Project.id, Project.user_id).where(Project.auto_summary.is_(None))
            )
            rows = result.all()
    except Exception:
        logger.warning("Project summary backfill: failed to query projects", exc_info=True)
        return

    if not rows:
        logger.info("Project summary backfill: all projects already have summaries")
        return

    logger.info("Project summary backfill: generating for %d projects", len(rows))
    for project_id, user_id in rows:
        await generate_project_summary(user_id, project_id)

    logger.info("Project summary backfill complete")
```

Also add `from fabledassistant.models.note import Note` at the top of `services/projects.py` (it already imports `Note`? — check; if not, add it).

- [ ] **Step 4: Trigger summary regen from `update_project()`**

In `services/projects.py`, after the final `return project` in `update_project()`:

```python
    import asyncio as _asyncio
    _asyncio.create_task(generate_project_summary(user_id, project_id))
    return project
```

- [ ] **Step 5: Trigger from note saves (debounced)**

In `services/notes.py`, in both `create_note()` and `update_note()`, after `await session.commit()` / `await session.refresh(note)`, add (only when `project_id` is set):

```python
    # Trigger project summary refresh if note belongs to a project (debounced: 1h)
    if note.project_id is not None:
        import asyncio as _asyncio
        from fabledassistant.services.projects import _should_refresh_summary, generate_project_summary
        if await _should_refresh_summary(note.user_id, note.project_id):
            _asyncio.create_task(generate_project_summary(note.user_id, note.project_id))
```

Add helper `_should_refresh_summary` to `services/projects.py`:

```python
async def _should_refresh_summary(user_id: int, project_id: int) -> bool:
    """Return True if summary is absent or more than 1 hour old."""
    from datetime import timedelta
    try:
        async with async_session() as session:
            result = await session.execute(
                select(Project.summary_updated_at).where(
                    Project.id == project_id, Project.user_id == user_id
                )
            )
            row = result.scalars().first()
        if row is None:
            return True
        return datetime.now(timezone.utc) - row > timedelta(hours=1)
    except Exception:
        return False
```

- [ ] **Step 6: Wire backfill into `app.py` startup**

In `app.py`, inside the `_delayed_backfill()` async function (after the `backfill_note_embeddings()` call):

```python
        try:
            from fabledassistant.services.projects import backfill_project_summaries
            await backfill_project_summaries()
        except Exception:
            logger.warning("Project summary backfill failed", exc_info=True)
```

- [ ] **Step 7: Run tests**

```bash
docker compose exec app pytest tests/test_rag_scoping.py -v
```

Expected: all PASS

- [ ] **Step 8: Commit**

```bash
git add src/fabledassistant/services/projects.py src/fabledassistant/services/notes.py src/fabledassistant/app.py tests/test_rag_scoping.py
git commit -m "feat(rag): project summarization background job with startup backfill"
```

---

### Task 7: New LLM tools — search_projects and set_rag_scope

**Files:**
- Modify: `src/fabledassistant/services/tools.py`
- Modify: `tests/test_rag_scoping.py`

- [ ] **Step 1: Write tests for the new tools**

Append to `tests/test_rag_scoping.py`:

```python
@pytest.mark.asyncio
async def test_search_projects_returns_ranked_results():
    """search_projects() ranks projects by text similarity to query."""
    from fabledassistant.services.tools import execute_tool

    mock_projects = [
        MagicMock(id=1, title="SciFi World", description="space opera", goal="", auto_summary="A story about aliens and humans"),
        MagicMock(id=2, title="Work Tasks", description="productivity", goal="", auto_summary="Task management for work"),
    ]

    with patch("fabledassistant.services.tools.list_projects", new_callable=AsyncMock, return_value=mock_projects):
        result = await execute_tool(1, "search_projects", {"query": "scifi aliens"})
        assert result["success"] is True
        assert result["type"] == "projects_list"
        # SciFi project should score higher
        projects = result["data"]["projects"]
        assert len(projects) > 0
        assert projects[0]["id"] == 1


@pytest.mark.asyncio
async def test_set_rag_scope_persists_to_db():
    """set_rag_scope() writes rag_project_id to conversation."""
    from fabledassistant.services.tools import execute_tool

    mock_project = MagicMock()
    mock_project.title = "SciFi World"

    with patch("fabledassistant.services.tools.get_project", new_callable=AsyncMock, return_value=mock_project), \
         patch("fabledassistant.services.tools.update_conversation", new_callable=AsyncMock, return_value=MagicMock()):
        result = await execute_tool(1, "set_rag_scope", {"project_id": 3}, conv_id=42)
        assert result["success"] is True
        assert result["type"] == "rag_scope_set"
        assert "SciFi World" in result["data"]["scope_label"]


@pytest.mark.asyncio
async def test_set_rag_scope_blocked_in_workspace():
    """set_rag_scope() returns error when workspace_project_id is set."""
    from fabledassistant.services.tools import execute_tool

    result = await execute_tool(1, "set_rag_scope", {"project_id": 3},
                                 conv_id=42, workspace_project_id=5)
    assert result["success"] is False
    assert "workspace" in result["error"].lower()
```

- [ ] **Step 2: Run tests — expect failures**

```bash
docker compose exec app pytest tests/test_rag_scoping.py::test_search_projects_returns_ranked_results -v
```

Expected: FAIL (`search_projects` not in `_CORE_TOOLS`)

- [ ] **Step 3: Add tool definitions to `_CORE_TOOLS` in `tools.py`**

In `services/tools.py`, add to the `_CORE_TOOLS` list alongside existing tool definitions:

```python
{
    "type": "function",
    "function": {
        "name": "search_projects",
        "description": "Search your projects by name or description to find which project to scope the conversation to. Returns ranked matches.",
        "parameters": {
            "type": "object",
            "properties": {
                "query": {"type": "string", "description": "Search query — project name, theme, or topic"},
            },
            "required": ["query"],
        },
    },
},
{
    "type": "function",
    "function": {
        "name": "set_rag_scope",
        "description": "Change the note search scope for this conversation. project_id=null means orphan notes only (default). project_id=-1 means all notes. A positive integer scopes to that project.",
        "parameters": {
            "type": "object",
            "properties": {
                "project_id": {
                    "type": ["integer", "null"],
                    "description": "Project ID to scope to, null for orphan-only, or -1 for all notes",
                },
            },
            "required": ["project_id"],
        },
    },
},
```

- [ ] **Step 4: Update `execute_tool()` signature**

Change:
```python
async def execute_tool(user_id: int, tool_name: str, arguments: dict) -> dict:
```

To:
```python
async def execute_tool(
    user_id: int,
    tool_name: str,
    arguments: dict,
    conv_id: int | None = None,
    workspace_project_id: int | None = None,
) -> dict:
```

- [ ] **Step 5: Add `search_projects` handler in `execute_tool()`**

Add the handler after the existing tool handlers (before the final `return {"success": False, "error": f"Unknown tool: {tool_name}"}`):

```python
    if tool_name == "search_projects":
        from difflib import SequenceMatcher
        from fabledassistant.services.projects import list_projects
        query = str(arguments.get("query", "")).lower().strip()
        if not query:
            return {"success": False, "error": "query is required"}
        projects = await list_projects(user_id)
        scored = []
        query_words = set(query.split())
        for p in projects:
            combined = f"{p.title} {p.description or ''} {p.auto_summary or ''}".lower()
            ratio = SequenceMatcher(None, query, combined).ratio()
            keyword_bonus = sum(0.05 for w in query_words if w in combined)
            score = min(ratio + keyword_bonus, 1.0)
            scored.append((score, p))
        scored.sort(key=lambda x: x[0], reverse=True)
        top = scored[:5]
        return {
            "success": True,
            "type": "projects_list",
            "data": {
                "projects": [
                    {
                        "id": p.id,
                        "title": p.title,
                        "summary_snippet": (p.auto_summary or p.description or "")[:200],
                        "score": round(score, 2),
                    }
                    for score, p in top
                ]
            },
        }

    if tool_name == "set_rag_scope":
        if workspace_project_id is not None:
            return {"success": False, "error": "Cannot change RAG scope in workspace view"}
        if conv_id is None:
            return {"success": False, "error": "No active conversation"}
        from fabledassistant.services.chat import update_conversation
        from fabledassistant.services.projects import get_project
        raw_id = arguments.get("project_id")
        # project_id can be null (None), -1, or positive int
        if raw_id is None:
            project_id_val = None
            scope_label = "Orphan notes only"
        elif int(raw_id) == -1:
            project_id_val = -1
            scope_label = "All notes"
        else:
            pid = int(raw_id)
            project = await get_project(user_id, pid)
            if project is None:
                return {"success": False, "error": f"Project {pid} not found"}
            project_id_val = pid
            scope_label = project.title
        await update_conversation(user_id, conv_id, rag_project_id=project_id_val)
        return {
            "success": True,
            "type": "rag_scope_set",
            "data": {
                "project_id": project_id_val,
                "scope_label": scope_label,
            },
        }
```

Also add `"rag_scope_set"` and `"projects_list"` to the `label` computed in `ToolCallCard.vue` (frontend Task 9).

- [ ] **Step 6: Run tests**

```bash
docker compose exec app pytest tests/test_rag_scoping.py -v
```

Expected: all PASS

- [ ] **Step 7: Commit**

```bash
git add src/fabledassistant/services/tools.py tests/test_rag_scoping.py
git commit -m "feat(rag): add search_projects and set_rag_scope LLM tools"
```

---

### Task 8: Thread conv_id through generation_task; emit new_rag_scope in SSE done

**Files:**
- Modify: `src/fabledassistant/services/generation_task.py`

- [ ] **Step 1: Thread `conv_id` and `workspace_project_id` into `execute_tool()` call**

Find the line (around 283) in `generation_task.py`:

```python
result = await execute_tool(user_id, tool_name, arguments)
```

Replace with:

```python
result = await execute_tool(
    user_id, tool_name, arguments,
    conv_id=conv_id,
    workspace_project_id=workspace_project_id,
)
```

`conv_id` and `workspace_project_id` are already in scope as parameters of `run_generation()`.

- [ ] **Step 2: Capture `new_rag_scope` and emit in done event**

Before the tool execution loop (around where `round_tool_calls = []` is initialized), add:

```python
new_rag_scope: int | None | bool = False  # False = "not changed"
new_rag_scope_label: str | None = None
```

After the `result = await execute_tool(...)` line, add:

```python
if result.get("type") == "rag_scope_set" and result.get("success"):
    new_rag_scope = result["data"]["project_id"]
    new_rag_scope_label = result["data"]["scope_label"]
```

Find the `done` event emission (line ~355):

```python
buf.append_event("done", {"done": True, "message_id": msg_id, "timing": timing})
```

Replace with:

```python
done_payload: dict = {"done": True, "message_id": msg_id, "timing": timing}
if new_rag_scope is not False:
    done_payload["new_rag_scope"] = new_rag_scope
    done_payload["new_rag_scope_label"] = new_rag_scope_label
buf.append_event("done", done_payload)
```

- [ ] **Step 3: Run full test suite**

```bash
docker compose exec app pytest tests/ -v 2>&1 | tail -30
```

Expected: all PASS

- [ ] **Step 4: Commit**

```bash
git add src/fabledassistant/services/generation_task.py
git commit -m "feat(rag): thread conv_id to execute_tool; emit new_rag_scope in SSE done event"
```

---

### Task 9: Frontend — types, chat store, scope chip in ChatView

**Files:**
- Modify: `frontend/src/types/chat.ts`
- Modify: `frontend/src/stores/chat.ts`
- Modify: `frontend/src/views/ChatView.vue`

- [ ] **Step 1: Update `Conversation` type in `types/chat.ts`**

Add `rag_project_id?: number | null` to the `Conversation` interface:

```typescript
export interface Conversation {
  id: number;
  title: string;
  model: string;
  message_count: number;
  rag_project_id?: number | null;
  created_at: string;
  updated_at: string;
}
```

- [ ] **Step 2: Add `ragProjectId` computed and `updateRagScope()` to chat store**

In `stores/chat.ts`, add after the `lastContextMeta` computed (around line 76):

```typescript
const ragProjectId = computed<number | null>(
  () => currentConversation.value?.rag_project_id ?? null
);

async function updateRagScope(convId: number, ragProjectId: number | null) {
  await apiPatch(`/api/chat/conversations/${convId}`, { rag_project_id: ragProjectId });
  if (currentConversation.value?.id === convId) {
    currentConversation.value.rag_project_id = ragProjectId;
  }
  const idx = conversations.value.findIndex((c) => c.id === convId);
  if (idx !== -1) conversations.value[idx].rag_project_id = ragProjectId;
}
```

In the SSE `done` handler (inside the `case "done":` block, after `s.streaming = false`), add:

```typescript
if (event.data.new_rag_scope !== undefined) {
  const newScope = event.data.new_rag_scope as number | null;
  if (currentConversation.value?.id === convId) {
    currentConversation.value.rag_project_id = newScope;
  }
  const idx2 = conversations.value.findIndex((c) => c.id === convId);
  if (idx2 !== -1) conversations.value[idx2].rag_project_id = newScope;
}
```

Add `ragProjectId` and `updateRagScope` to the store's return object.

- [ ] **Step 3: Add `rag_scope_set` and `projects_list` labels to ToolCallCard**

In `frontend/src/components/ToolCallCard.vue`, add to the `label` computed:

```typescript
case "rag_scope_set": return "Scope changed";
case "projects_list": return "Projects";
```

- [ ] **Step 4: Replace sidebar ProjectSelector with scope chip in ChatView**

Remove the existing RAG scope section in the sidebar:
```vue
<!-- RAG project scope -->
<div class="rag-scope-section">
  <label class="rag-scope-label">Scope notes to project</label>
  <ProjectSelector v-model="ragProjectId" />
  <p v-if="ragProjectId" class="rag-scope-hint">RAG search restricted to this project</p>
</div>
```

Load projects list in the script:
```typescript
import { listProjects } from "@/api/client";  // or use apiGet

interface ScopeProject { id: number; title: string; }
const scopeProjects = ref<ScopeProject[]>([]);
const scopeDropdownOpen = ref(false);

onMounted(async () => {
  try {
    const res = await apiGet<{ projects: ScopeProject[] }>("/api/projects?status=active");
    scopeProjects.value = res.projects;
  } catch { /* non-fatal */ }
});

const scopeLabel = computed(() => {
  const id = chatStore.ragProjectId;
  if (id === null || id === undefined) return "Orphan notes";
  if (id === -1) return "All notes";
  return scopeProjects.value.find((p) => p.id === id)?.title ?? `Project ${id}`;
});

async function setScope(value: number | null) {
  scopeDropdownOpen.value = false;
  if (!convId.value) return;
  await chatStore.updateRagScope(convId.value, value);
}
```

Remove the old `ragProjectId` ref (it's now a computed from the store). Update `sendMessage()` to use `chatStore.ragProjectId` instead of the local ref.

Add the scope chip above the message input in the template (inside `.chat-main`):

```vue
<div class="scope-chip-row">
  <div class="scope-chip" :class="{ active: scopeDropdownOpen }" @click="scopeDropdownOpen = !scopeDropdownOpen">
    <span class="scope-dot"></span>
    <span class="scope-label">{{ scopeLabel }}</span>
    <span class="scope-chevron">▾</span>
  </div>
  <div v-if="scopeDropdownOpen" class="scope-dropdown">
    <button class="scope-option" @click="setScope(null)">Orphan notes only</button>
    <button
      v-for="p in scopeProjects"
      :key="p.id"
      class="scope-option"
      @click="setScope(p.id)"
    >{{ p.title }}</button>
    <button class="scope-option" @click="setScope(-1)">All notes</button>
  </div>
</div>
```

Add scoped styles:

```css
.scope-chip-row {
  position: relative;
  padding: 0 1rem 0.35rem;
  display: flex;
  align-items: center;
}
.scope-chip {
  display: inline-flex;
  align-items: center;
  gap: 0.3rem;
  padding: 0.2rem 0.6rem;
  border-radius: 999px;
  border: 1px solid var(--color-border);
  background: var(--color-bg-secondary);
  font-size: 0.75rem;
  color: var(--color-text-muted);
  cursor: pointer;
  user-select: none;
  transition: border-color 0.15s;
}
.scope-chip:hover, .scope-chip.active {
  border-color: var(--color-primary);
  color: var(--color-primary);
}
.scope-dot {
  width: 6px;
  height: 6px;
  border-radius: 50%;
  background: var(--color-primary);
  flex-shrink: 0;
}
.scope-dropdown {
  position: absolute;
  top: calc(100% - 0.1rem);
  left: 1rem;
  z-index: 100;
  background: var(--color-bg-card);
  border: 1px solid var(--color-border);
  border-radius: var(--radius-md);
  min-width: 180px;
  box-shadow: 0 4px 16px var(--color-shadow, rgba(0,0,0,0.2));
  overflow: hidden;
}
.scope-option {
  display: block;
  width: 100%;
  text-align: left;
  padding: 0.45rem 0.75rem;
  background: none;
  border: none;
  font-size: 0.82rem;
  color: var(--color-text);
  cursor: pointer;
  font-family: inherit;
  transition: background 0.1s;
}
.scope-option:hover {
  background: color-mix(in srgb, var(--color-primary) 8%, transparent);
  color: var(--color-primary);
}
```

- [ ] **Step 5: TypeScript check**

```bash
cd frontend && npx vue-tsc --noEmit 2>&1 | head -30
```

Expected: no errors

- [ ] **Step 6: Commit**

```bash
git add frontend/src/types/chat.ts frontend/src/stores/chat.ts frontend/src/views/ChatView.vue frontend/src/components/ToolCallCard.vue
git commit -m "feat(rag): scope chip in chat, ragProjectId in store, SSE done handler"
```

---

### Task 10: Full test run and final commit

**Files:** none new

- [ ] **Step 1: Run full test suite**

```bash
docker compose exec app pytest tests/ -v 2>&1 | tail -30
```

Expected: all PASS

- [ ] **Step 2: TypeScript check**

```bash
cd frontend && npx vue-tsc --noEmit
```

Expected: no errors

- [ ] **Step 3: Final commit**

```bash
git add -A
git commit -m "feat(rag): RAG scoping — orphan-only default, project summaries, search_projects + set_rag_scope tools, scope chip UI"
```