FabledScribe/docs/specs/2026-03-25-briefing-improvements-design.md

# Briefing Service Improvements — Design Spec

**Date:** 2026-03-25
**Status:** Approved
**Scope:** Web-first (no Android changes in this cycle)

---

## Problem Statement

The daily briefing has several usability issues:

1. **Task repetition** — tasks are restated identically every day regardless of whether anything changed, making the briefing feel stale and hard to scan.
2. **RSS repetition** — the same news stories resurface across days with no mechanism to learn what the user cares about.
3. **No path to sources** — news items are summarised in prose with no link to the original article.
4. **Stale weather** — if the weather cache is outdated, the briefing silently uses old data rather than failing gracefully. The current prose weather format is also hard to scan.
5. **No feedback loop** — there is no way to teach the briefing what topics are interesting or uninteresting.

---

## Approach

**Pre-processing pipeline with explicit state tracking.** Rather than relying on the synthesis LLM to handle deduplication and filtering, we add deterministic pre-processing steps before synthesis runs. Each concern is isolated: task change detection, RSS topic classification and filtering, weather staleness gating. The synthesis LLM receives pre-filtered, structured input and focuses on tone and flow.

---

## Data Model

### Migration: `0028_add_briefing_improvements`

**`rss_items` table — two new columns**

```sql
ALTER TABLE rss_items ADD COLUMN IF NOT EXISTS topics TEXT[] DEFAULT '{}';
ALTER TABLE rss_items ADD COLUMN IF NOT EXISTS classified_at TIMESTAMPTZ;
```

`topics` stores LLM-assigned topic tags (e.g. `["technology", "ai"]`). `classified_at` is NULL until classification runs, allowing backfill queries. The `RssFeed` / `RssItem` SQLAlchemy model (`models/rss_feed.py`) must also be updated to add these two mapped columns and expose them in `to_dict()`.

**`messages` table — new metadata column**

```sql
ALTER TABLE messages ADD COLUMN IF NOT EXISTS metadata JSONB;
```

The briefing pipeline populates this when it creates the compiled message. The frontend reads it when loading the conversation to render the `WeatherCard` and attach reaction buttons. No SSE events are needed — structured data travels with the message record.

Schema stored in `metadata`:
```json
{
  "weather": {
    "location": "Berlin",
    "fetched_at": "2026-03-25T06:00:00Z",
    "current_temp": 12,
    "condition": "Partly Cloudy",
    "today_high": 16,
    "today_low": 8,
    "yesterday_high": 14,
    "yesterday_low": 9,
    "forecast": [
      {"day": "Wed", "condition": "Sunny", "high": 18, "low": 10},
      {"day": "Thu", "condition": "Cloudy", "high": 14, "low": 9}
    ]
  },
  "rss_item_ids": [42, 17, 89, 103, 55]
}
```

If weather is unavailable, `metadata.weather` is `null` and the card renders a failure placeholder. The `Message` SQLAlchemy model (`models/conversation.py`) must be updated to add the `metadata` mapped column.

**New table: `rss_item_reactions`**

```sql
CREATE TABLE IF NOT EXISTS rss_item_reactions (
    id          SERIAL PRIMARY KEY,
    user_id     INTEGER NOT NULL REFERENCES users(id) ON DELETE CASCADE,
    rss_item_id INTEGER NOT NULL REFERENCES rss_items(id) ON DELETE CASCADE,
    reaction    TEXT NOT NULL CHECK (reaction IN ('up', 'down')),
    created_at  TIMESTAMPTZ DEFAULT NOW(),
    UNIQUE (user_id, rss_item_id)
);
CREATE INDEX IF NOT EXISTS ix_rss_item_reactions_user_id ON rss_item_reactions(user_id);
```

One reaction per user per item. A second click on the same button removes the reaction; clicking the opposite button flips it.

**New table: `briefing_task_snapshot`**

```sql
CREATE TABLE IF NOT EXISTS briefing_task_snapshot (
    id            SERIAL PRIMARY KEY,
    user_id       INTEGER NOT NULL REFERENCES users(id) ON DELETE CASCADE,
    task_id       INTEGER NOT NULL REFERENCES notes(id) ON DELETE CASCADE,
    snapshot_hash TEXT NOT NULL,
    last_briefed  TIMESTAMPTZ DEFAULT NOW(),
    UNIQUE (user_id, task_id)
);
CREATE INDEX IF NOT EXISTS ix_briefing_task_snapshot_user_id ON briefing_task_snapshot(user_id);
```

`snapshot_hash` is `SHA-256(status + priority + due_date + title)`. The pipeline diffs current task state against these rows to detect what has changed since the last briefing.

**Settings keys (existing key-value store — no new table)**

- `briefing_include_topics` — JSON array of topic strings to prioritise
- `briefing_exclude_topics` — JSON array of topics to hard-exclude from briefings

---

## Pipeline Changes

### Pre-processing Stage (new, runs before parallel gather)

Three sequential steps added to `services/briefing_pipeline.py`. Note: `_gather_internal` currently serialises tasks as dicts without `id`. It must be updated to include `task_id` in each serialised task dict (the `Note` ORM object has `.id` available) so the post-briefing snapshot upsert has the required FK value.

**1. Task change detection**

For each of the user's current tasks, compute `SHA-256(status + priority + due_date + title)` and compare against `briefing_task_snapshot`. Split into:
- `changed_tasks` — new hash or no snapshot row (included fully in briefing)
- `unchanged_count` — integer count passed to the synthesis prompt as context

The synthesis prompt receives `changed_tasks` and the instruction: "N tasks are unchanged since the last briefing — acknowledge this briefly rather than listing them."

**2. RSS item filtering**

Load the user's `briefing_include_topics` and `briefing_exclude_topics` settings, plus reaction history (last 30 days, aggregated per topic as a net score). Score each recent classified item:
- Hard-remove items tagged with any excluded topic
- Boost items tagged with any included or positively-reacted topic
- Penalise items from negatively-reacted topics
- Sort by score, take top 10

Items with `classified_at IS NULL` pass through unfiltered (new feeds not yet classified) and are queued for background classification.

**3. Weather staleness gate**

Check `weather_cache.fetched_at`. If older than 24 hours: skip weather entirely, set `weather_unavailable = True`. The frontend renders a `WeatherCard` placeholder in the failure state. If fresh: pass the forecast JSON (including `past_days=1` data) to the pipeline for card rendering.

### RSS Classification (background, triggered at fetch time)

When `services/rss.py` stores new items, it queues a fire-and-forget async task to classify them. Classification is a fast, non-streaming LLM call processing batches of up to 10 items:

```
Classify each news item into 1-3 topics from this vocabulary:
technology, science, politics, business, health, environment,
local, entertainment, sports, other, [user_defined_topics]
Return JSON: {"item_id": ["topic1", "topic2"]}
```

The vocabulary is extended with the user's declared preference topics so custom interests can be matched. Results are written to `rss_items.topics` and `classified_at`.

Model: the user's `default_model` setting (same as chat). If the LLM is unavailable or classification fails, the item is stored with `topics = []` and `classified_at` left NULL — it will be retried the next time new items are fetched. No retry loop; classification is best-effort.

### Post-briefing Stage (new, runs after `post_message()` returns)

1. **Upsert task snapshots** — upsert `briefing_task_snapshot` rows for all tasks included in this briefing so the next run can diff against current state.
2. **Populate message metadata** — the `metadata` dict (`weather` + `rss_item_ids`) is assembled during pre-processing and passed through to `post_message()`, which writes it to the `Message.metadata` column. No separate post-step is needed — the metadata is stored atomically with the message.

---

## Weather Card

The weather section is no longer generated as prose by the synthesis LLM. Instead:

- `services/weather.py` is updated to request `past_days=1` from Open-Meteo, including yesterday's high/low in the same API response.
- The pipeline parses the forecast into the `metadata.weather` schema (defined in the Data Model section) and stores it on the `Message` record when `post_message()` is called.
- `BriefingView.vue` reads `message.metadata.weather` when loading the conversation and renders `WeatherCard.vue` above the message text if the field is present.
- The synthesis LLM's weather section is suppressed entirely — the prompt instructs it to skip weather since it is handled by the card.

**`WeatherCard.vue` displays:**
- Location name and "as of" timestamp
- Current temperature and condition
- Today's high / low
- Yesterday's high / low with delta ("3° warmer than yesterday")
- Compact 3–5 day forecast strip (day name, condition, high/low)

**Failure state:** If `metadata.weather` is `null`, the same card position renders a muted placeholder: "Weather data unavailable — will retry at next slot."

---

## News Cards

### Format

The synthesis LLM is instructed to format each included news item as:

```markdown
**[Headline text](source_url)**
*Outlet Name · Day Month*
One or two sentence summary of the story.
```

No prose wrapper between cards. The synthesis prompt must explicitly instruct the LLM to **present news items in the exact order provided** — `metadata.rss_item_ids` records this order and the frontend maps reaction buttons positionally. Reordering by the LLM would break the mapping.

The briefing message structure becomes:
1. Greeting / task summary
2. `WeatherCard` (rendered from `message.metadata.weather`, not prose)
3. News cards (markdown blocks with links)
4. Calendar / other sections

### Reaction Buttons

`BriefingView.vue` reads `message.metadata.rss_item_ids` when loading the conversation. The ordered list of IDs maps directly to the news cards in the rendered message (cards appear in synthesis output in the same order). A 👍 / 👎 pair is rendered below each card. Reaction buttons are only shown in the briefing view — not in message history exports.

Clicking a reaction:
1. Optimistic UI update (button enters selected state immediately)
2. `POST /api/briefing/rss-reactions` — `{rss_item_id, reaction}`
3. Backend validates ownership: joins through `rss_items → rss_feeds` to confirm `rss_feeds.user_id = g.user.id` before upserting
4. Upserts into `rss_item_reactions` — same reaction removes it, opposite flips it

**New endpoints in `routes/briefing.py`:**
- `POST /api/briefing/rss-reactions` — upsert or remove reaction (ownership-checked)
- `DELETE /api/briefing/rss-reactions/:item_id` — explicit removal (useful for MCP/external API consumers that cannot use the toggle behaviour of POST)

---

## Topic Preferences UI

**Settings → Briefing tab — new "News Preferences" subsection** (added below existing RSS feed management):

Two chip-input fields using the existing `TagInput.vue` component:
- **Interested in** → `briefing_include_topics` setting
- **Not interested in** → `briefing_exclude_topics` setting

A collapsed hint lists the standard topic vocabulary so users know valid terms. Custom terms are accepted — the RSS classifier will attempt to match them.

Saved via the existing `PUT /api/settings/:key` endpoint.

---

## MCP Tool Additions

New file: `fable-mcp/fable_mcp/tools/briefing.py`

Three tools registered in `server.py`:

| Tool | Description |
|------|-------------|
| `add_rss_feed(url, title=None, category=None)` | Adds a feed to the user's RSS list. `title` is optional — the feed title is auto-populated from feed metadata after the first fetch, but an override can be passed. Returns the created feed object. |
| `list_rss_feeds()` | Returns current feed list with id, title, url, category, last_fetched_at. |
| `remove_rss_feed(feed_id)` | Removes a feed by ID. |

These call existing RSS endpoints in `routes/briefing.py` via `FableClient`. No new backend routes required.

---

## New Backend Files

| File | Purpose |
|------|---------|
| `services/briefing_preferences.py` | Load/compute topic preference weights; apply to RSS item scoring |
| `services/rss_classifier.py` | Batch LLM classification of RSS items; background task management |

## Modified Backend Files

| File | Changes |
|------|---------|
| `services/briefing_pipeline.py` | Add pre-processing and post-briefing stages; carry `task_id` through serialised task dicts; pass `metadata` dict to `post_message()` |
| `services/rss.py` | Trigger background classification after storing new items |
| `services/weather.py` | Add `past_days=1` to Open-Meteo request; expose parsed yesterday data |
| `routes/briefing.py` | Add `POST/DELETE /api/briefing/rss-reactions` endpoints |
| `models/rss_feed.py` | Add `topics` and `classified_at` mapped columns to `RssItem`; expose in `to_dict()` |
| `models/conversation.py` | Add `metadata` JSONB mapped column to `Message`; update `to_dict()` to include `metadata` in the returned dict |
| `services/briefing_conversations.py` | Extend `post_message(conversation_id, role, content)` signature to accept an optional `metadata: dict \| None = None` parameter; pass it to the `Message(...)` constructor |

## New Frontend Files

| File | Purpose |
|------|---------|
| `components/WeatherCard.vue` | Weather display card (current, today, yesterday delta, 3-5 day strip, failure state) |

## Modified Frontend Files

| File | Changes |
|------|---------|
| `views/BriefingView.vue` | Read `message.metadata.weather` on conversation load → render `WeatherCard.vue` above message text; read `message.metadata.rss_item_ids` → attach reaction buttons to news cards in order |
| `views/SettingsView.vue` | Add "News Preferences" subsection with two `TagInput` fields |
| `api/client.ts` | Add `postRssReaction()`, `deleteRssReaction()` helpers |

---

## Out of Scope

- Android companion app changes (web-first; parity deferred)
- Full numeric scoring system (Approach C) — can evolve to this once reaction data accumulates
- Push notification integration for briefing reactions