# Unified Lookup Tool & Wikipedia Integration

## Goal

Replace the fragmented `search_web` tool with a single `lookup` tool that checks Wikipedia first and falls back to SearXNG web search. Add Wikipedia as an additional source in the research pipeline. Result: one lightweight tool for factual questions (always available, no config required), and richer research output.

## Architecture

Two changes to the search/knowledge stack:

1. **New `lookup` tool** replaces `search_web`. Tries Wikipedia REST API summary endpoint first (~200ms, reliable, no config). Falls back to SearXNG + trafilatura article fetch when Wikipedia misses and SearXNG is configured. Always available (no `requires` field).

2. **Wikipedia sources in research pipeline.** During sub-query execution, `wiki_search` runs alongside `_search_searxng`. Wikipedia articles merge into the source pool and get deduplicated by URL.

Shared Wikipedia logic lives in a new `wikipedia.py` service module.

## Components

### `src/fabledassistant/services/wikipedia.py` (new)

Two async functions:

**`wiki_summary(query: str) -> dict | None`**
- Direct title lookup via `https://en.wikipedia.org/api/rest_v1/page/summary/{title}`
- Returns `{"title": str, "extract": str, "url": str}` on hit
- Returns `None` on 404, disambiguation pages (`"type": "disambiguation"`), network errors, or empty extracts
- 5-second timeout
- User-Agent: `"FabledAssistant/1.0 (https://fabledsword.com)"`

**`wiki_search(query: str, limit: int = 3) -> list[dict]`**
- Search via `https://en.wikipedia.org/w/api.php?action=query&list=search&srsearch={query}&srlimit={limit}&format=json`
- For each search result, fetch its summary via the summary endpoint to get the extract
- Returns `[{"title": str, "extract": str, "url": str}, ...]`
- Returns `[]` on any failure
- Same timeout and User-Agent as above

### `src/fabledassistant/services/tools/web.py` (modified)

**Remove:** `search_web_tool`

**Add:** `lookup_tool`

```
@tool(
    name="lookup",
    description="Look up a topic, concept, or factual question. Returns a concise
                 answer from Wikipedia or web sources. Use for definitions,
                 explanations, 'what is X', 'how does Y work'. For comprehensive
                 written reports saved as notes, use research_topic instead.",
    parameters={
        "query": {"type": "string", "description": "The topic or question to look up"},
    },
    required=["query"],
)
```

No `requires` field — always available.

**Logic:**
1. Call `wiki_summary(query)`
2. If Wikipedia returns a result: return `{"success": True, "type": "lookup", "source": "wikipedia", "data": {"title": ..., "extract": ..., "url": ...}}`
3. If Wikipedia misses and `Config.searxng_enabled()`:
   - Call `_search_searxng(query)` to get search results
   - Fetch top 1-2 result URLs via `_fetch_full_article` (from `rss.py`, trafilatura-based)
   - Return `{"success": True, "type": "lookup", "source": "web", "data": {"query": ..., "results": [...], "content": ...}}`
4. If Wikipedia misses and no SearXNG: return `{"success": True, "type": "lookup", "source": "none", "data": {"query": ..., "message": "No results found. You can answer from your own knowledge."}}`

### `src/fabledassistant/services/research.py` (modified)

**In Step 2 (parallel search):**
- For each sub-query, run `wiki_search(query, limit=1)` concurrently with `_search_searxng(query)`
- Merge Wikipedia results into the per-query result list

**In Step 3 (deduplication):**
- When deduplicating URLs, Wikipedia URLs (`wikipedia.org`) are checked against SearXNG results
- If a Wikipedia article URL already appears in SearXNG results, skip the duplicate

**Wikipedia article content for synthesis:**
- The `extract` from `wiki_search` is used as the source content (no additional fetch needed, unlike SearXNG URLs which require `fetch_url_content`)
- This means Wikipedia sources are available immediately without an HTTP fetch step

## Error Handling

- All Wikipedia API failures (network, timeout, malformed JSON) return `None`/`[]` silently
- `lookup` never raises — always returns a response the model can work with
- In the research pipeline, Wikipedia is purely additive; its failure never degrades existing SearXNG-based research
- Disambiguation pages are detected via `"type": "disambiguation"` in the summary response and treated as a miss

## Testing

### `tests/test_wikipedia.py` (new)

- `test_wiki_summary_returns_extract` — mock successful summary response, verify return shape
- `test_wiki_summary_returns_none_on_404` — mock 404, verify `None`
- `test_wiki_summary_returns_none_on_disambiguation` — mock disambiguation response, verify `None`
- `test_wiki_search_returns_results` — mock search API + summary fetches, verify list
- `test_wiki_search_returns_empty_on_failure` — mock network error, verify `[]`

### `tests/test_lookup_tool.py` (new)

- `test_lookup_wikipedia_hit` — mock `wiki_summary` returning data, verify tool returns wikipedia source
- `test_lookup_wikipedia_miss_searxng_fallback` — mock `wiki_summary` returning None, SearXNG returning results + article fetch, verify web source
- `test_lookup_wikipedia_miss_no_searxng` — mock both missing, verify graceful "no results" response
- `test_lookup_always_available` — verify the tool appears in `get_tools_for_user` regardless of SearXNG config

### `tests/test_research_pipeline.py` (add to existing)

- `test_research_includes_wikipedia_sources` — mock `wiki_search` alongside SearXNG, verify Wikipedia results appear in source pool

All tests mock HTTP calls — no live API hits.

## What Doesn't Change

- `read_article` tool — stays as-is (explicit URL fetch, different purpose)
- `research_topic` tool definition — stays as-is (same name, description, parameters)
- `generation_task.py` research interception — stays as-is
- `search_images` tool — stays as-is
- `_search_searxng` and `_search_searxng_images` — stay as-is
- `_fetch_full_article` in `rss.py` — stays as-is, reused by `lookup` for SearXNG fallback