FabledScribe/docs/superpowers/plans/2026-04-05-article-reading.md

Article Reading Implementation Plan

For agentic workers: REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (- [ ]) syntax for tracking.

Goal: Add a read_article tool so the LLM can fetch any URL, fix the history builder so tool context survives follow-up turns, redesign the Discuss button to inject article content as a persisted tool exchange, and remove the RSS content character cap.

Architecture: Four independent changes executed in dependency order: (1) content cap removal, (2) read_article tool, (3) history builder fix (prerequisite for everything persisting across follow-ups), (4) Discuss endpoint + frontend. Each task is independently committable.

Tech Stack: Python/Quart, SQLAlchemy async, trafilatura (already installed), httpx (already installed), Vue 3 + TypeScript frontend.

---

File map

Action	Path	Responsibility
Modify	src/fabledassistant/services/rss.py	Remove CONTENT_MAX_CHARS truncation
Modify	src/fabledassistant/services/tools.py	Add _URL_TOOLS list, add read_article to get_tools_for_user, add handler in execute_tool
Modify	src/fabledassistant/routes/chat.py	Fix history builder to replay tool_calls
Modify	src/fabledassistant/services/chat.py	Add tool_calls parameter to add_message
Modify	src/fabledassistant/routes/briefing.py	Add POST /api/briefing/articles/<item_id>/discuss endpoint
Modify	frontend/src/views/BriefingView.vue	Replace discussArticle() to call new endpoint
Modify	tests/test_rss_service.py	Update truncation test, add no-truncation test
Create	tests/test_article_reading.py	Tests for read_article tool and history builder

---

Task 1: Remove RSS content cap

Files:

• Modify: src/fabledassistant/services/rss.py:17-18,83,213
• Modify: tests/test_rss_service.py:19-26

The CONTENT_MAX_CHARS = 50_000 constant and all uses of [:CONTENT_MAX_CHARS] are removed. Trafilatura extracts only article body text, so content is naturally bounded.

• Step 1: Update the truncation test to assert no truncation

In tests/test_rss_service.py, replace the existing test_extract_item_truncates_content test:

def test_extract_item_does_not_truncate_content():
    """extract_item() should store content without truncation."""
    from fabledassistant.services.rss import extract_item
    long_text = "x" * 100_000
    entry = MagicMock()
    entry.get = lambda k, d="": {"summary": long_text, "title": "", "link": "", "id": "g"}.get(k, d)
    entry.content = []
    entry.published_parsed = None
    item = extract_item(entry)
    assert len(item["content"]) == 100_000

• Step 2: Run the test to confirm it fails

cd /home/bvandeusen/Nextcloud/Projects/fabledassistant
make test ARGS="tests/test_rss_service.py::test_extract_item_does_not_truncate_content -v"

Expected: FAIL (current code truncates to 50_000).

• Step 3: Remove CONTENT_MAX_CHARS from rss.py

In src/fabledassistant/services/rss.py:

Remove lines 17–18:

# Safety cap on stored content — effectively unlimited for typical articles
CONTENT_MAX_CHARS = 50_000

Change line 83 from:

    content = _html_to_text(content)[:CONTENT_MAX_CHARS]

to:

    content = _html_to_text(content)

Change line 213 from:

                            item.content = full_text[:CONTENT_MAX_CHARS]

to:

                            item.content = full_text

• Step 4: Run all rss tests

make test ARGS="tests/test_rss_service.py -v"

Expected: all pass. The test_extract_item_truncates_content test name no longer exists (replaced in Step 1).

• Step 5: Commit

git add src/fabledassistant/services/rss.py tests/test_rss_service.py
git commit -m "feat(rss): remove article content character cap"

---

Task 2: Add read_article tool

Files:

• Modify: src/fabledassistant/services/tools.py
• Create: tests/test_article_reading.py

The tool uses _fetch_full_article from rss.py (lazy import inside execute_tool to avoid circular dependencies). Added unconditionally to all users via a new _URL_TOOLS list.

• Step 1: Write failing tests

Create tests/test_article_reading.py:

import json
import pytest
from unittest.mock import AsyncMock, patch


@pytest.mark.asyncio
async def test_read_article_success():
    """read_article tool returns article content on success."""
    from fabledassistant.services.tools import execute_tool
    with patch(
        "fabledassistant.services.rss._fetch_full_article",
        new=AsyncMock(return_value="Article text here."),
    ):
        result = await execute_tool(
            user_id=1,
            tool_name="read_article",
            arguments={"url": "https://example.com/article"},
        )
    assert result["success"] is True
    assert result["type"] == "article_content"
    assert result["url"] == "https://example.com/article"
    assert result["content"] == "Article text here."
    assert result["truncated"] is False


@pytest.mark.asyncio
async def test_read_article_fetch_failure():
    """read_article tool returns success=False when fetch returns None."""
    from fabledassistant.services.tools import execute_tool
    with patch(
        "fabledassistant.services.rss._fetch_full_article",
        new=AsyncMock(return_value=None),
    ):
        result = await execute_tool(
            user_id=1,
            tool_name="read_article",
            arguments={"url": "https://example.com/bad"},
        )
    assert result["success"] is False
    assert "Could not fetch" in result["error"]


@pytest.mark.asyncio
async def test_read_article_truncates_at_40k():
    """read_article tool truncates content at 40_000 chars and sets truncated=True."""
    from fabledassistant.services.tools import execute_tool
    long_content = "x" * 50_000
    with patch(
        "fabledassistant.services.rss._fetch_full_article",
        new=AsyncMock(return_value=long_content),
    ):
        result = await execute_tool(
            user_id=1,
            tool_name="read_article",
            arguments={"url": "https://example.com/long"},
        )
    assert result["success"] is True
    assert len(result["content"]) == 40_000
    assert result["truncated"] is True


@pytest.mark.asyncio
async def test_read_article_empty_url():
    """read_article tool returns success=False when url is empty."""
    from fabledassistant.services.tools import execute_tool
    result = await execute_tool(
        user_id=1,
        tool_name="read_article",
        arguments={"url": ""},
    )
    assert result["success"] is False

• Step 2: Run tests to confirm they fail

make test ARGS="tests/test_article_reading.py -v"

Expected: all 4 fail with "read_article not handled" or AttributeError.

• Step 3: Add _URL_TOOLS list and register it in get_tools_for_user

In src/fabledassistant/services/tools.py, add the _URL_TOOLS list immediately after the _SEARCH_TOOLS block (around line 836):

_URL_TOOLS = [
    {
        "type": "function",
        "function": {
            "name": "read_article",
            "description": (
                "Fetch and read the full text of a web page or article from a URL. "
                "Use when the user shares a URL and wants you to read it, or to get "
                "the full content of a linked page. "
                "Do NOT use search_web for URLs — use this tool instead."
            ),
            "parameters": {
                "type": "object",
                "properties": {
                    "url": {"type": "string", "description": "The URL to fetch and read"}
                },
                "required": ["url"],
            },
        },
    }
]

In get_tools_for_user (around line 1034), add _URL_TOOLS unconditionally after _CORE_TOOLS:

async def get_tools_for_user(user_id: int) -> list[dict]:
    """Build the tool list for a user based on their configured integrations."""
    tools = list(_CORE_TOOLS)
    tools.extend(_URL_TOOLS)
    tools.extend(_RAG_TOOLS)
    tools.extend(_ENTITY_TOOLS)
    if await is_caldav_configured(user_id):
        tools.extend(_CALDAV_TOOLS)
    if Config.searxng_enabled():
        tools.extend(_SEARCH_TOOLS)
        tools.extend(_RESEARCH_TOOLS)
        tools.extend(_IMAGE_TOOLS)
    logger.debug("User %d: %d tools available", user_id, len(tools))
    return tools

• Step 4: Add read_article handler in execute_tool

In src/fabledassistant/services/tools.py, in the execute_tool function, find the elif tool_name == "search_web": block (around line 1771). Add the new handler immediately before it:

        elif tool_name == "read_article":
            from fabledassistant.services.rss import _fetch_full_article
            url = arguments.get("url", "").strip()
            if not url:
                return {"success": False, "error": "No URL provided"}
            content = await _fetch_full_article(url)
            if not content:
                return {"success": False, "error": f"Could not fetch article content from {url}"}
            _TOOL_CONTENT_CAP = 40_000
            truncated = len(content) > _TOOL_CONTENT_CAP
            return {
                "success": True,
                "type": "article_content",
                "url": url,
                "content": content[:_TOOL_CONTENT_CAP],
                "truncated": truncated,
            }

• Step 5: Run the tests

make test ARGS="tests/test_article_reading.py -v"

Expected: all 4 pass.

• Step 6: Run full test suite

make test

Expected: all pass.

• Step 7: Commit

git add src/fabledassistant/services/tools.py tests/test_article_reading.py
git commit -m "feat(tools): add read_article tool using trafilatura extraction"

---

Task 3: Fix history builder

Files:

• Modify: src/fabledassistant/routes/chat.py:162-166
• Modify: tests/test_article_reading.py (add history builder tests)

The loop that builds history for run_generation currently drops tool_calls. This fix replays the full tool exchange so the LLM sees prior tool results on follow-up turns.

• Step 1: Add history builder tests

Append to tests/test_article_reading.py:

def test_history_builder_plain_messages():
    """Messages without tool_calls are added as {role, content} unchanged."""
    import json
    messages = [
        type("M", (), {"role": "system", "content": "sys", "tool_calls": None})(),
        type("M", (), {"role": "user", "content": "hello", "tool_calls": None})(),
        type("M", (), {"role": "assistant", "content": "hi", "tool_calls": None})(),
    ]
    history = _build_history(messages)
    assert history == [
        {"role": "user", "content": "hello"},
        {"role": "assistant", "content": "hi"},
    ]


def test_history_builder_with_tool_calls():
    """Messages with tool_calls emit an assistant entry + tool result entries."""
    import json
    tool_calls_data = [
        {
            "function": "read_article",
            "arguments": {"url": "https://example.com"},
            "result": {"success": True, "content": "Article text"},
        }
    ]
    messages = [
        type("M", (), {"role": "user", "content": "read this", "tool_calls": None})(),
        type("M", (), {
            "role": "assistant",
            "content": "",
            "tool_calls": tool_calls_data,
        })(),
        type("M", (), {"role": "user", "content": "follow up", "tool_calls": None})(),
    ]
    history = _build_history(messages)
    assert history[0] == {"role": "user", "content": "read this"}
    assert history[1]["role"] == "assistant"
    assert history[1]["tool_calls"] == [
        {"function": {"name": "read_article", "arguments": {"url": "https://example.com"}}}
    ]
    assert history[2] == {"role": "tool", "content": json.dumps({"success": True, "content": "Article text"})}
    assert history[3] == {"role": "user", "content": "follow up"}


def _build_history(messages):
    """Inline copy of the fixed history builder for testing."""
    import json
    history = []
    for msg in messages:
        if msg.role == "system":
            continue
        msg_dict = {"role": msg.role, "content": msg.content or ""}
        if msg.tool_calls:
            msg_dict["tool_calls"] = [
                {"function": {"name": tc["function"], "arguments": tc["arguments"]}}
                for tc in msg.tool_calls
            ]
            history.append(msg_dict)
            for tc in msg.tool_calls:
                history.append({"role": "tool", "content": json.dumps(tc.get("result", {}))})
        else:
            history.append(msg_dict)
    return history

• Step 2: Run the tests to confirm they pass

(These tests use _build_history defined inline — they test the logic directly, not the route. They should pass immediately.)

make test ARGS="tests/test_article_reading.py::test_history_builder_plain_messages tests/test_article_reading.py::test_history_builder_with_tool_calls -v"

Expected: both pass.

• Step 3: Apply the fix to chat.py

In src/fabledassistant/routes/chat.py, replace lines 162–166:

    # Build history from existing messages (excluding system and the placeholder)
    history = []
    for msg in conv.messages:
        if msg.role != "system":
            history.append({"role": msg.role, "content": msg.content})

with:

    # Build history from existing messages (excluding system and the placeholder).
    # Tool calls from prior turns are replayed as assistant tool_call + tool result
    # messages so the LLM retains tool context on follow-up turns.
    history = []
    for msg in conv.messages:
        if msg.role == "system":
            continue
        msg_dict = {"role": msg.role, "content": msg.content or ""}
        if msg.tool_calls:
            msg_dict["tool_calls"] = [
                {"function": {"name": tc["function"], "arguments": tc["arguments"]}}
                for tc in msg.tool_calls
            ]
            history.append(msg_dict)
            for tc in msg.tool_calls:
                history.append({"role": "tool", "content": json.dumps(tc.get("result", {}))})
        else:
            history.append(msg_dict)

json is already imported at the top of chat.py.

• Step 4: Run full test suite

make test

Expected: all pass.

• Step 5: Commit

git add src/fabledassistant/routes/chat.py tests/test_article_reading.py
git commit -m "fix(chat): replay tool_calls in history so tool context survives follow-up turns"

---

Task 4: Extend add_message to accept tool_calls

Files:

• Modify: src/fabledassistant/services/chat.py:183-207

The Discuss endpoint (Task 5) needs to store a synthetic assistant message with tool_calls. The existing add_message doesn't support this parameter.

• Step 1: Update add_message signature and body

In src/fabledassistant/services/chat.py, replace the add_message function (lines 183–207):

async def add_message(
    conversation_id: int,
    role: str,
    content: str,
    context_note_id: int | None = None,
    status: str | None = None,
    tool_calls: list | None = None,
) -> Message:
    async with async_session() as session:
        kwargs: dict = dict(
            conversation_id=conversation_id,
            role=role,
            content=content,
            context_note_id=context_note_id,
        )
        if status is not None:
            kwargs["status"] = status
        if tool_calls is not None:
            kwargs["tool_calls"] = tool_calls
        msg = Message(**kwargs)
        session.add(msg)
        # Touch conversation updated_at
        conv = await session.get(Conversation, conversation_id)
        if conv:
            conv.updated_at = datetime.now(timezone.utc)
        await session.commit()
        await session.refresh(msg)
        return msg

• Step 2: Run full test suite

make test

Expected: all pass (existing callers only use positional/keyword args that are unchanged).

• Step 3: Commit

git add src/fabledassistant/services/chat.py
git commit -m "feat(chat): add tool_calls parameter to add_message"

---

Task 5: Add Discuss endpoint and update frontend

Files:

• Modify: src/fabledassistant/routes/briefing.py
• Modify: frontend/src/views/BriefingView.vue

New route: POST /api/briefing/articles/<item_id>/discuss. Fetches stored article from DB, stores a synthetic read_article tool exchange plus the user message, then triggers generation. Frontend replaces the inline-content approach with a call to this endpoint.

• Step 1: Add the discuss endpoint to briefing.py

At the top of src/fabledassistant/routes/briefing.py, add these imports (after the existing imports):

from fabledassistant.models.rss_feed import RssItem, RssFeed
from fabledassistant.services.chat import add_message, get_conversation
from fabledassistant.services.generation_buffer import GenerationState, create_buffer, get_buffer
from fabledassistant.services.generation_task import run_generation
from fabledassistant.services.settings import get_setting

Note: get_setting and asyncio are already imported. Add only what is missing.

Then add the new route at the end of briefing.py (before any final lines), after the list_news route:

@briefing_bp.route("/articles/<int:item_id>/discuss", methods=["POST"])
@_REQUIRE
async def discuss_article(item_id: int):
    """Pre-load a briefing article as a read_article tool exchange and trigger generation."""
    uid = g.user.id
    data = await request.get_json() or {}
    conv_id = data.get("conv_id")
    if not conv_id:
        return jsonify({"error": "conv_id is required"}), 400

    # Verify article belongs to this user (via feed ownership)
    async with async_session() as session:
        result = await session.execute(
            select(RssItem).join(RssFeed, RssItem.feed_id == RssFeed.id)
            .where(RssItem.id == item_id, RssFeed.user_id == uid)
        )
        item = result.scalar_one_or_none()
    if item is None:
        return jsonify({"error": "Article not found"}), 404

    # Verify conversation belongs to this user
    conv = await get_conversation(uid, conv_id)
    if conv is None:
        return jsonify({"error": "Conversation not found"}), 404

    # Reject if generation already running
    existing = get_buffer(conv_id)
    if existing and existing.state == GenerationState.RUNNING:
        return jsonify({"error": "Generation already in progress"}), 409

    article_content = item.content or ""

    # Store synthetic assistant message: read_article was already called with stored content
    synthetic_tool_calls = [
        {
            "function": "read_article",
            "arguments": {"url": item.url},
            "result": {
                "success": True,
                "type": "article_content",
                "url": item.url,
                "content": article_content,
                "truncated": False,
            },
        }
    ]
    await add_message(conv_id, "assistant", "", status="complete", tool_calls=synthetic_tool_calls)

    # Store user message
    await add_message(conv_id, "user", "Please summarize and discuss this article.")

    # Reload conversation so history includes the two new messages
    conv = await get_conversation(uid, conv_id)

    # Build history (using the fixed builder from chat.py logic — duplicated here)
    history = []
    for msg in conv.messages:
        if msg.role == "system":
            continue
        msg_dict = {"role": msg.role, "content": msg.content or ""}
        if msg.tool_calls:
            msg_dict["tool_calls"] = [
                {"function": {"name": tc["function"], "arguments": tc["arguments"]}}
                for tc in msg.tool_calls
            ]
            history.append(msg_dict)
            for tc in msg.tool_calls:
                history.append({"role": "tool", "content": json.dumps(tc.get("result", {}))})
        else:
            history.append(msg_dict)

    model = await get_setting(uid, "default_model", "") or ""
    from fabledassistant.config import Config as _Config
    if not model:
        model = _Config.OLLAMA_MODEL

    # Create placeholder assistant message and generation buffer
    assistant_msg = await add_message(conv_id, "assistant", "", status="generating")
    try:
        buf = create_buffer(conv_id, assistant_msg.id)
    except RuntimeError:
        return jsonify({"error": "Generation already in progress"}), 409

    asyncio.create_task(run_generation(
        buf, history, model,
        uid, conv_id, conv.title,
        "Please summarize and discuss this article.",
        think=True,
    ))

    return jsonify({"assistant_message_id": assistant_msg.id, "status": "generating"}), 202

• Step 2: Run full test suite

make test

Expected: all pass.

• Step 3: Update discussArticle in BriefingView.vue

In frontend/src/views/BriefingView.vue, replace the discussArticle function:

async function discussArticle(item: NewsItem) {
  if (!todayConvId.value || chatStore.streaming) return
  if (!isToday.value) selectedConvId.value = todayConvId.value
  await nextTick(() => {
    document.querySelector('.briefing-center')?.scrollIntoView({ behavior: 'smooth', block: 'nearest' })
  })
  try {
    await apiPost<{ assistant_message_id: number }>(
      `/api/briefing/articles/${item.id}/discuss`,
      { conv_id: todayConvId.value },
    )
  } catch {
    return
  }
  // Reload conversation so the new messages appear (including the generating placeholder),
  // then reconnect to the SSE stream using the existing reconnectIfGenerating helper.
  await chatStore.fetchConversation(todayConvId.value)
  await chatStore.reconnectIfGenerating(todayConvId.value)
}

reconnectIfGenerating is already exported from useChatStore. It finds the assistant message in status="generating" state and connects to the SSE stream automatically. No changes to chat.ts are needed.

• Step 4: TypeScript check

cd /home/bvandeusen/Nextcloud/Projects/fabledassistant
npm --prefix frontend run type-check

Expected: no errors.

• Step 5: Commit

git add src/fabledassistant/routes/briefing.py frontend/src/views/BriefingView.vue frontend/src/stores/chat.ts
git commit -m "feat(briefing): add discuss endpoint and update frontend to use persisted article context"

---

Task 6: Final verification

• Step 1: Run full test suite

cd /home/bvandeusen/Nextcloud/Projects/fabledassistant
make test

Expected: all tests pass.

• Step 2: TypeScript check

npm --prefix frontend run type-check

Expected: no errors.

• Step 3: Push

git push origin dev