# Agentic Briefing — Design Spec

**Date:** 2026-04-10
**Status:** Proposed
**Author:** bvandeusen + Claude

---

## Problem

The current briefing pipeline hallucinates calendar events, tasks, and news items that do not exist in the user's actual data. Observed examples from production:

- A morning briefing asserting "your dentist appointment is still in progress" when no such event existed
- An 8am check-in mentioning "a quick meeting at 10:30 AM" with no backing calendar entry
- A midday check-in inventing "a team huddle at 2:30 PM" and "the Q2 budget draft due by Friday"
- The model calling `search_images` in response to a clarifying question about a fabricated meeting

These are not bugs in data retrieval — the data gathering code (`_gather_internal`, `_gather_external`) returns correct values. They are a structural consequence of how the briefing context is assembled.

## Root cause — the "no receipt" problem

The current `run_compilation` pipeline does this:

1. Python code gathers data (tasks, events, weather, news) from the database
2. Python formats the data into a structured text blob as the user-role message: `TODAY'S EVENTS: ...`, `DUE TODAY: ...`, etc.
3. The LLM is called **once** with `[system_prompt, user_prompt]` and produces prose
4. Only the prose reply is written to the conversation. **The underlying data is never persisted in the conversation history.**

When the user later chats in the briefing conversation, the chat endpoint loads the full conversation history — which contains the model's *prose* from earlier but not the data that prose was derived from. The model's own prior output becomes the only source of "truth" available for follow-up questions. If that prose asserted a fact (real or hallucinated), the model has no way to distinguish it from ground truth when generating the next reply, and it will double down.

Compounding factors:

- **Empty sections are silently omitted.** If `calendar_events` is empty, the user prompt contains no `TODAY'S EVENTS:` line at all. The model has no explicit "zero events" signal — combined with an imperative system prompt ("note calendar events and tasks"), it interprets the silence as "I should mention some" and fabricates.
- **Scheduled slot injections append synthetic turns.** `run_slot_injection` writes a fake `[Midday briefing update]` user message and the assistant reply into the persistent conversation. By evening, the chat history contains three separate briefings, each potentially with errors, all treated as equal-weight context on follow-up.
- **The `search_images` tool is available during briefing chat**, with a negative-instruction description ("Not for factual questions"). Small and mid-sized models frequently ignore negative guidance in tool descriptions and call the tool anyway.

## Solution — agentic briefing (the receipt model)

Replace the one-shot synthesis with a tool-call loop. The briefing is no longer "a text blob synthesized from pre-gathered data." It is **a scheduled agent run**: the LLM is given a system prompt, a curated set of read-only data tools, and a trigger ("generate the morning briefing"). The model must call tools to see what exists. Every tool call and tool result becomes part of the conversation history, where it lives as a permanent, structured receipt.

### Why this fixes the hallucination

When the model calls `list_events(today) → []`, that empty array is now a persistent message in the conversation. On a follow-up question like "what meeting?", the chat endpoint loads the conversation and the model sees its own tool result from the morning showing no events. Answering "you had a meeting at 10:30" would require the model to directly contradict a tool result sitting two messages back — something LLMs are much more reliable at avoiding than contradicting their own prior prose.

In short: **the model cannot fabricate what has a visible receipt proving it does not exist.**

### What the model sees — before vs after

**Before** — the model sees one structured blob and produces prose. The blob is discarded.

```
[system] You are a personal assistant... weave together what matters...
[user]   Date: 2026-04-10
         WEATHER: Home — partly cloudy, 8–16°C
         DUE TODAY: Fix briefing hallucinations
         OVERDUE (2 tasks): ...
[assistant] Morning! Looks like a quiet day with two overdue items...
```

**After** — the model is given tools and iterates to ground truth.

```
[system] You are the user's personal assistant giving their morning briefing.
         Use tools to see what's relevant. Only mention things you learned from
         tool calls. If a tool returns nothing, say so honestly.
[user]   Generate the morning briefing.
[assistant] tool_call: list_tasks(filter="due_today")
[tool]   [{"id": 81, "title": "Fix briefing hallucinations", ...}]
[assistant] tool_call: list_tasks(filter="overdue")
[tool]   [{"id": 42, ...}, {"id": 43, ...}]
[assistant] tool_call: list_events(today)
[tool]   []
[assistant] tool_call: get_weather(location="home")
[tool]   {"description": "partly cloudy", "temp_min": 8, "temp_max": 16}
[assistant] tool_call: get_rss_items(max=8)
[tool]   [...]
[assistant] Morning! Partly cloudy today, 8 to 16 — nothing on the calendar
         so it's a clean run at the desk. Two things to keep in mind...
```

The conversation now contains verifiable receipts: `list_events(today)` returned `[]`, and that result sits in context forever (until pruned). Follow-up questions operate against those receipts.

## Architecture

### Existing infrastructure (reused, not rebuilt)

The codebase already has the agentic primitives — they're used for regular chat:

- **`llm.py::stream_chat_with_tools`** — the streaming tool-use loop that talks to Ollama with a `tools` parameter and yields tool-call chunks
- **`generation_task.py::_stream_with_retry`** — wraps `stream_chat_with_tools` with retry-on-500 behavior for cold-model races
- **`tools.py`** — defines 40+ tool schemas and an execution dispatcher

The briefing bypasses all of this and calls a one-shot `_llm_synthesise` helper. The refactor is mainly "route briefings through the same pipeline regular chat already uses."

### New modules

**`briefing_tools.py`** — a small wrapper exposing a curated read-only subset of `tools.py` for briefing runs. This is an **explicit allowlist**, not a blocklist, so newly-added tools must be opted in:

| Tool | Purpose |
|---|---|
| `list_tasks` (with filter args) | See what's actionable today, overdue, high priority |
| `list_events` (today / upcoming) | Know what's on the calendar |
| `get_weather` | Current/forecast weather |
| `get_rss_items` | Pull news themes filtered by user preferences |
| `list_projects` | Understand project context |
| `search_projects` | Surface active project summaries |
| `list_notes` (recent) | Capture follow-ups from yesterday |

**Explicitly omitted** from the briefing tool set:

- `search_images`, `search_web`, `research_topic`, `read_article` — external search is not a briefing concern, and `search_images` is the source of the "Peter Kyle Science Secretary" image-search bug
- All `create_*`, `update_*`, `delete_*` — briefings are read-only; a scheduled background job must not decide to mutate the user's data on its own
- `set_rag_scope`, `calculate` — not relevant to briefing content

### New briefing function

**`briefing_pipeline.py::run_agentic_briefing(user_id, slot, model, conversation_id)`** replaces `run_compilation`'s body (and eventually `run_slot_injection`). Internally:

1. Build a slot-specific system prompt (see below)
2. Load the curated briefing tools from `briefing_tools.py`
3. Seed messages with `[system, user]` where the user message is a simple trigger like `"Generate the morning briefing."`
4. Enter a tool-call loop (max 8 iterations):
   - Call `stream_chat_with_tools`
   - If the model returns `tool_calls`, execute them via the existing dispatcher, append tool results, continue
   - If the model returns a final assistant message with no pending tool calls, break
5. Return `(final_prose, full_message_list, metadata)`

The full message list is important: it's written to the conversation along with the final prose, so the tool-call receipts become part of the persistent record.

### Slot-specific system prompts

Compilation (full morning briefing):

```
You are the user's personal assistant giving their full morning briefing.
Use the tools available to see what's actually relevant today — tasks due,
overdue items, events on the calendar, weather, news themes, project state
— and weave it into a warm, natural-sounding summary.

Rules:
- Call tools to see the data. Never assert facts you didn't learn from a tool.
- If a tool returns nothing (no events today, no overdue tasks), say so
  honestly. Don't fabricate items to fill space.
- Write flowing prose. No markdown, no headers, no bullet points.
- Aim for 6–10 sentences. Skip topics that have nothing interesting.
- Close on one or two concrete, actionable suggestions.

User profile (for tone and preferences):
{profile_body}
```

Check-ins (midday, afternoon):

```
You are the user's personal assistant giving a brief {slot} check-in.
Use tools to see what's changed since this morning. Focus on progress
and what's still unaddressed.

Rules:
- Call tools to see current state. Never assert facts without tool results.
- If nothing meaningful has changed, say so briefly — don't invent progress.
- 3–5 sentences, natural prose, no markdown.
```

### Conversation hygiene — removing the fake user messages

The current scheduler appends two fake messages on every slot injection:

```python
await post_message(conv.id, "user", f"[{slot.title()} briefing update]")
await post_message(conv.id, "assistant", text)
```

Under the agentic model, we drop the fake `"user"` message entirely. Slot updates become plain assistant messages tagged with `metadata.briefing_slot = "midday"`. The chat endpoint's message loader is updated to filter these when building the LLM context on follow-ups, so a user chatting in the briefing conversation doesn't see three earlier briefings smashed into their history. The UI continues to show them as visible timeline entries.

(This is the Option A filter from the earlier discussion — a small, surgical change compared to migrating briefings to a separate sidecar table. Sidecar storage remains a possible future step.)

### Ollama setup compatibility

The existing Ollama deployment uses non-parallel mode across two GPUs, with a concern about context duplication between cards. This is the correct setup for agentic briefings:

- Each tool-call iteration shares the same KV cache on the same GPU, so appending tool results is cheap
- There is no race where a second iteration could land on a different card with a different cache state
- The trade-off is serialization: a briefing in progress will block a concurrent user chat request until it finishes, but scheduled briefings are rare enough (4× per day) that this is acceptable

If GPU contention becomes a problem later, the right lever is pinning specific models to specific GPUs (e.g., background tasks on GPU 1, interactive models on GPU 0) — not enabling parallelism.

## Cost & trade-offs

- **Latency:** a briefing now makes 5–7 inference calls (one per tool-call decision plus the final prose) instead of 1. On a local Ollama with a 7B model, expect 15–40s per briefing vs the current 5–10s. Acceptable for a background job; if it becomes painful at the user-facing check-in slots, investigate letting the model batch independent tool calls in a single turn (Ollama supports this).
- **Model requirements:** the default model must be reliable at tool calling. `qwen2.5:7b`, `llama3.1:8b`, `mistral-small-3:24b`, and similar handle it well. Models in the ≤3B class typically fail — they either emit no tool calls and return empty prose, or hallucinate invalid tool arguments. If a user's `default_model` is too small, agentic mode should fall back to legacy mode with a log warning.
- **Context growth:** tool results bloat the conversation. At 8 tool calls per compilation × 4 slots per day, a daily briefing conversation can reach 20-30 KB of JSON-heavy history. Fine for a day; aged briefings should be archived/rolled up after 7 days.
- **Tool subset drift:** someone adds a new mutating tool to `tools.py` and forgets to update the briefing allowlist. Mitigated by the allowlist model — the default for new tools is "not exposed to briefings."
- **Infinite loop safety:** a buggy model could tool-call forever. Hard cap at 8 iterations, log a warning if hit, return whatever prose was last produced (or a fallback message).

## Migration path

Ship incrementally, each step independently reversible:

**PR 1 — Agentic compilation behind a feature flag.** Add `briefing_tools.py`, add `run_agentic_briefing`, add a per-user setting `briefing_mode: "legacy" | "agentic"` (default legacy). Route only the 4am compilation through the new path when the flag is set. Keep slot-injection on the legacy path. Enable the flag on the author's account first, validate output quality over several days, then flip the default for all users. No DB migration required — the setting lives in the existing `settings` table.

**PR 2 — Agentic slot injections + conversation hygiene.** Migrate midday/afternoon check-ins to the same pipeline. Remove the fake `[Midday briefing update]` user-role message; slot updates become plain assistant messages tagged with `metadata.briefing_slot`. Add a chat-message-loader filter that excludes slot-tagged messages from the LLM context on follow-ups (they remain visible in the UI).

**PR 3 — UI polish.** Collapsed tool-call status row in the briefing card ("✓ checked calendar · ✓ looked at tasks · ✓ pulled weather"), expanding to show tool results on click. Tool-result cards (weather, news, task list) rendered inline where useful.

**PR 4 (optional) — Sidecar storage for briefing snapshots.** If the chat-filter approach in PR 2 feels too hacky, migrate briefings out of the conversations table and into a dedicated `briefing_snapshots` table. Frontend renders them as pinned timeline cards separate from chat. Larger refactor; defer until PR 1–3 prove the approach works.

## Secondary win — tightening the main chat system prompt

The regular chat is already agentic (it uses `stream_chat_with_tools`), but its system prompt does not explicitly require the model to ground factual claims in tool results. The prompt discipline introduced for briefings — *"Never assert facts you didn't learn from a tool. If a tool returns nothing, say so honestly."* — is worth applying to the main chat system prompt in a follow-up PR. The mechanism already works; only the framing needs tightening.

## Out of scope

- The Android Flutter client's failure to render `search_images` tool-result cards. This is a separate rendering gap in the mobile app and does not affect the server-side fix. Tracked separately.
- Re-evaluating the Ollama parallelism setting. The current non-parallel config is correct for this work.
- Replacing the background model for title/summary/observation extraction. That model's role is unrelated to briefings.