feat(briefing): cache + map-reduce article context for rich discuss chats
The Discuss button on news cards was producing one-shot replies because the model got the whole trafilatura blob dropped into history with a canned "summarize and discuss this article" prompt — no length guard, no prep, no invitation to converse. Large articles got silently truncated by Ollama; small articles got a tepid reply. This reworks discuss_article around a three-layer cache: context_prepared → content_full → fresh trafilatura fetch First click on a small article fetches once, writes through to both caches, and passes the body straight into the synthetic read_article tool-result. First click on a large article additionally runs a parallel map step (services/article_context.py) that chunks the body on paragraph boundaries, summarizes each ~8k chunk to ~300 words of dense factual prose via the background model, and concatenates the summaries under section headers — all pinned to num_ctx=16384 so the map step doesn't itself fall victim to silent truncation. Repeat clicks on either path skip straight to the chat turn. The canned summary prompt is replaced with a conversational seed that invites the user into an actual discussion rather than a one-shot synopsis, matching the goal of "have a conversation about an article, not just read it." discuss_topic is intentionally left untouched — it's the multi-article aggregation path and needs a separate rework. Follow-up task will decide whether to retire it or rework it on the cached-context approach. Closes task #106. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
@@ -34,6 +34,34 @@ def _html_to_text(html: str) -> str:
|
||||
return html
|
||||
|
||||
|
||||
async def get_or_fetch_full_article(item: RssItem) -> str | None:
|
||||
"""Return the full article body, fetching+caching on miss.
|
||||
|
||||
Checks ``item.content_full`` first — populated either by the enrichment
|
||||
pass at feed-ingest time or by a previous discuss-click. On miss, fetches
|
||||
via ``_fetch_full_article`` and writes through. Returns ``None`` only if
|
||||
the fetch itself fails; ``item.content_full == ""`` is still a cache hit.
|
||||
|
||||
Callers must pass an RssItem attached to an open session if they want
|
||||
the write-through to persist — otherwise the fetched text is returned
|
||||
but the cache stays empty and the next click will re-fetch.
|
||||
"""
|
||||
if item.content_full is not None:
|
||||
return item.content_full
|
||||
if not item.url:
|
||||
return None
|
||||
text = await _fetch_full_article(item.url)
|
||||
if text is None:
|
||||
return None
|
||||
async with async_session() as session:
|
||||
fresh = await session.get(RssItem, item.id)
|
||||
if fresh is not None:
|
||||
fresh.content_full = text
|
||||
fresh.content_fetched_at = datetime.now(timezone.utc)
|
||||
await session.commit()
|
||||
return text
|
||||
|
||||
|
||||
async def _fetch_full_article(url: str) -> str | None:
|
||||
"""Fetch a URL and extract its main article text via trafilatura.
|
||||
|
||||
@@ -209,6 +237,11 @@ async def fetch_and_cache_feed(feed_id: int, url: str) -> int:
|
||||
item = await session.get(RssItem, item_id)
|
||||
if item:
|
||||
item.content = full_text
|
||||
# Populate the discuss-click cache too so the
|
||||
# first click skips straight to the map-reduce
|
||||
# step without re-fetching.
|
||||
item.content_full = full_text
|
||||
item.content_fetched_at = datetime.now(timezone.utc)
|
||||
await session.commit()
|
||||
await upsert_rss_item_embedding(
|
||||
item_id, feed_user_id, item.title or "", item.content
|
||||
|
||||
Reference in New Issue
Block a user