Compare commits

..

6 Commits

Author SHA1 Message Date
bvandeusen fd885c1bc8 Merge pull request 'Release v26.04.13.3' (#31) from dev into main 2026-04-14 03:36:09 +00:00
bvandeusen 8205590f8d feat(briefing): cache + map-reduce article context for rich discuss chats
The Discuss button on news cards was producing one-shot replies because
the model got the whole trafilatura blob dropped into history with a
canned "summarize and discuss this article" prompt — no length guard, no
prep, no invitation to converse. Large articles got silently truncated by
Ollama; small articles got a tepid reply.

This reworks discuss_article around a three-layer cache:

  context_prepared  →  content_full  →  fresh trafilatura fetch

First click on a small article fetches once, writes through to both
caches, and passes the body straight into the synthetic read_article
tool-result. First click on a large article additionally runs a parallel
map step (services/article_context.py) that chunks the body on paragraph
boundaries, summarizes each ~8k chunk to ~300 words of dense factual
prose via the background model, and concatenates the summaries under
section headers — all pinned to num_ctx=16384 so the map step doesn't
itself fall victim to silent truncation. Repeat clicks on either path
skip straight to the chat turn.

The canned summary prompt is replaced with a conversational seed that
invites the user into an actual discussion rather than a one-shot
synopsis, matching the goal of "have a conversation about an article,
not just read it."

discuss_topic is intentionally left untouched — it's the multi-article
aggregation path and needs a separate rework. Follow-up task will decide
whether to retire it or rework it on the cached-context approach.

Closes task #106.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-13 20:52:00 -04:00
bvandeusen 939b910372 Merge pull request 'Release v26.04.13.2' (#30) from dev into main 2026-04-13 23:00:33 +00:00
bvandeusen 70cea78c2f fix(llm): default generate_completion num_ctx to Config.OLLAMA_NUM_CTX
Non-streaming generate_completion was the only LLM entry point that
didn't default num_ctx — stream_chat and stream_chat_with_tools both
fall back to Config.OLLAMA_NUM_CTX (16384). When a caller omitted the
argument, Ollama silently used the model's default window (~4k on
qwen3) and truncated the prompt.

That footgun was masked by fallback paths in the research pipeline:
_generate_outline's prompt carries ~12 sources × 2000 chars (~6k
tokens) of source material plus a system prompt, so the prompt got
chopped, the model never saw the sources, JSON parsing failed twice,
and run_research_pipeline dropped into the single-note "monolith"
fallback (research.py:251). The user reported chat 215 producing such
a monolith note for a multi-source research topic.

Two-layer fix:
- Default num_ctx to Config.OLLAMA_NUM_CTX inside generate_completion,
  matching the streaming entry points. Any current or future caller
  that forgets the argument stops silently losing input.
- Pin num_ctx=16384 explicitly in _generate_outline and
  _generate_executive_summary with comments pointing at the failure
  mode, so a refactor of the generate_completion default can't
  silently regress the research pipeline.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-13 18:20:58 -04:00
bvandeusen 4e4dbb8783 fix(chat): feed title model raw turns instead of post-build_context messages
_generate_title was receiving the full messages list from build_context,
which prepends RAG snippets, RSS excerpts, URL content, and briefing
article dumps INTO the user-role message string. The role=="user" filter
inside _generate_title then handed that composite blob (capped at 300
chars) to gemma3:4b as "the user's message", so the background model
was titling conversations based on article excerpts instead of what the
user actually typed — producing wildly wrong titles like "Briefing
Profile Preferences & Schedule" for a plain calendar query. See #109.

Pass the raw history + user_content + assistant reply instead.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-13 17:15:39 -04:00
bvandeusen e4e1d1da49 fix(tz): interpret calendar and briefing dates in user's local timezone
Two related bugs where the server defaulted naive datetimes to UTC instead
of the configured user timezone, causing all-day events to land on the
previous day and briefings to "disappear" at UTC midnight.

- New services/tz.py helpers: get_user_tz, user_today, user_briefing_date
  (the briefing day flips at 4am local to align with the compilation slot,
  so the 00:00-04:00 local window still shows yesterday's briefing until
  the new one is generated).
- calendar create/list/update tools now parse naive datetimes in the
  user's TZ before converting to UTC for storage, and tool descriptions
  tell the model to pass plain local dates.
- briefing_conversations.get_or_create_today_conversation and the
  reset-today route use user_briefing_date so the in-progress briefing
  doesn't get replaced at 19:00 NY / UTC midnight.
- _run_profile_closeout targets user-local "yesterday" for consistency.

Regression tests added for the TZ helpers and the calendar tool.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-13 15:35:27 -04:00
15 changed files with 707 additions and 45 deletions
@@ -0,0 +1,34 @@
"""Add content_full and context_prepared caches to rss_items.
Revision ID: 0038
Revises: 0037
Create Date: 2026-04-13
"""
from __future__ import annotations
import sqlalchemy as sa
from alembic import op
revision = "0038"
down_revision = "0037"
branch_labels = None
depends_on = None
def upgrade() -> None:
op.add_column("rss_items", sa.Column("content_full", sa.Text(), nullable=True))
op.add_column(
"rss_items",
sa.Column("context_prepared", sa.Text(), nullable=True),
)
op.add_column(
"rss_items",
sa.Column("content_fetched_at", sa.DateTime(timezone=True), nullable=True),
)
def downgrade() -> None:
op.drop_column("rss_items", "content_fetched_at")
op.drop_column("rss_items", "context_prepared")
op.drop_column("rss_items", "content_full")
+11
View File
@@ -46,6 +46,17 @@ class RssItem(Base):
published_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True)
# Truncated to 2000 chars to keep DB size reasonable
content: Mapped[str] = mapped_column(Text, default="")
# Full trafilatura-extracted article body, populated lazily on first
# discuss-click / enrichment pass. Nullable — most items never get this
# cached. Expires naturally with the item (90-day retention).
content_full: Mapped[str | None] = mapped_column(Text, nullable=True)
# Map-reduced conversation-ready context derived from content_full. See
# services/article_context.py — populated on first discuss click so
# repeat clicks skip both the fetch and the LLM map step.
context_prepared: Mapped[str | None] = mapped_column(Text, nullable=True)
content_fetched_at: Mapped[datetime | None] = mapped_column(
DateTime(timezone=True), nullable=True
)
fetched_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True), default=lambda: datetime.now(timezone.utc)
)
+39 -9
View File
@@ -330,10 +330,13 @@ async def reset_today_briefing():
driven from the MCP when iterating on prompts. Pair with
``POST /api/briefing/trigger`` to immediately regenerate.
"""
from datetime import date as _date
from sqlalchemy import delete as _delete
today = _date.today()
from fabledassistant.services.tz import user_briefing_date
# User-local briefing day (flips at 4am local), not ``date.today()`` —
# see services/tz.py for rationale.
today = await user_briefing_date(g.user.id)
async with async_session() as session:
result = await session.execute(
select(Conversation).where(
@@ -529,8 +532,28 @@ async def discuss_article(item_id: int):
if get_buffer(conv_id) is not None:
return jsonify({"error": "Generation already in progress"}), 409
from fabledassistant.services.rss import _fetch_full_article
article_content = await _fetch_full_article(item.url) or item.content or ""
# Three-layer cache: context_prepared (post-map-reduce) → content_full
# (raw trafilatura) → fresh fetch. Only the first miss pays the fetch
# cost; only a large uncached article pays the map-reduce cost. Repeat
# clicks on the same article skip straight to the chat turn.
from fabledassistant.services.article_context import prepare_article_context
from fabledassistant.services.rss import get_or_fetch_full_article
model = await get_setting(uid, "default_model", "") or ""
if item.context_prepared:
article_content = item.context_prepared
else:
raw_body = await get_or_fetch_full_article(item) or item.content or ""
article_content = await prepare_article_context(
item.title or "", item.url, raw_body, model,
)
if article_content:
async with async_session() as session:
fresh = await session.get(RssItem, item.id)
if fresh is not None:
fresh.context_prepared = article_content
await session.commit()
# Store synthetic assistant message with read_article tool result
synthetic_tool_calls = [{
@@ -546,8 +569,17 @@ async def discuss_article(item_id: int):
}]
await add_message(conv_id, "assistant", "", status="complete", tool_calls=synthetic_tool_calls)
# Store user message
await add_message(conv_id, "user", "Please summarize and discuss this article.")
# Conversational seed — invites a real discussion rather than asking for
# a one-shot summary. The model sees the article context in the tool
# result above and responds to this user turn as the start of an ongoing
# conversation the user will steer with follow-ups.
discuss_prompt = (
"I want to talk about this article. Start with a substantive summary "
"of what it's arguing and the key evidence it uses, then tell me what "
"stood out to you or seems worth pushing back on. I'll ask follow-ups "
"from there."
)
await add_message(conv_id, "user", discuss_prompt)
# Reload conversation with fresh messages to build history
conv = await get_conversation(uid, conv_id)
@@ -569,15 +601,13 @@ async def discuss_article(item_id: int):
else:
history.append(msg_dict)
model = await get_setting(uid, "default_model", "") or ""
assistant_msg = await add_message(conv_id, "assistant", "", status="generating")
buf = create_buffer(conv_id, assistant_msg.id)
asyncio.create_task(run_generation(
buf, history, model,
uid, conv_id, conv.title or "",
"Please summarize and discuss this article.",
discuss_prompt,
))
return jsonify({"assistant_message_id": assistant_msg.id, "status": "generating"}), 202
@@ -0,0 +1,161 @@
"""Prepare article bodies as conversation-ready context.
Used by the briefing ``discuss-article`` flow. A raw trafilatura extraction
is often too large to drop whole into a chat history without eating the
context window, so this module runs a map-reduce step over oversized
articles and returns a compact, structured context that still preserves the
article's meaning across sections.
Small articles pass through unchanged — map-reduce only fires when the raw
body exceeds CHAR_BUDGET. The output is cached on ``rss_items.context_prepared``
by the caller, so repeat discuss-clicks on the same article skip this work
entirely.
"""
from __future__ import annotations
import asyncio
import logging
import re
from fabledassistant.services.llm import generate_completion
logger = logging.getLogger(__name__)
# ~12k tokens at 4 chars/token. Comfortably under OLLAMA_NUM_CTX=16384
# with room left for system prompt, chat history, and the assistant reply.
CHAR_BUDGET = 48_000
# Chunk size for the map step on oversized articles. Overlap preserves
# context across paragraph boundaries that happen to land mid-sentence.
CHUNK_CHARS = 8_000
CHUNK_OVERLAP = 400
_PARA_SPLIT = re.compile(r"\n\s*\n")
def _chunk_by_paragraph(body: str) -> list[str]:
"""Split ``body`` into chunks of up to CHUNK_CHARS, respecting paragraphs.
Paragraphs longer than CHUNK_CHARS are split mid-paragraph as a last
resort. Adjacent chunks share CHUNK_OVERLAP chars of trailing text so
a sentence straddling the boundary stays readable on both sides.
"""
paragraphs = [p.strip() for p in _PARA_SPLIT.split(body) if p.strip()]
chunks: list[str] = []
current: list[str] = []
current_len = 0
for para in paragraphs:
para_len = len(para)
if para_len > CHUNK_CHARS:
if current:
chunks.append("\n\n".join(current))
current, current_len = [], 0
for i in range(0, para_len, CHUNK_CHARS - CHUNK_OVERLAP):
chunks.append(para[i : i + CHUNK_CHARS])
continue
if current_len + para_len + 2 > CHUNK_CHARS and current:
chunks.append("\n\n".join(current))
tail = current[-1][-CHUNK_OVERLAP:] if current else ""
current = [tail, para] if tail else [para]
current_len = len(tail) + para_len + (2 if tail else 0)
else:
current.append(para)
current_len += para_len + 2
if current:
chunks.append("\n\n".join(current))
return chunks
async def _summarize_chunk(title: str, chunk: str, index: int, total: int, model: str) -> str:
"""Map-step summary of one article chunk.
Aims for ~300 words of dense, factual prose — not bullet points — so the
downstream chat model can quote from it naturally.
"""
messages = [
{
"role": "system",
"content": (
"You are summarizing one section of a larger article so a downstream "
"conversation model can discuss the full article without having to read "
"every word.\n\n"
"Requirements:\n"
"- 250350 words of dense factual prose\n"
"- Preserve specific claims, numbers, names, and quotes\n"
"- Do NOT editorialize or add analysis\n"
"- Do NOT use bullet points or headings\n"
"- Do NOT say 'this section' or 'this article' — write content, not meta"
),
},
{
"role": "user",
"content": (
f"Article: {title}\n"
f"Section {index + 1} of {total}:\n\n{chunk}"
),
},
]
try:
# Pin num_ctx — same rationale as services/research.py:66. A large
# chunk plus system prompt can push well past the default window;
# silent truncation here would drop the tail of the chunk without
# any error, producing a misleading summary.
raw = await generate_completion(
messages, model, max_tokens=600, num_ctx=16384
)
return raw.strip()
except Exception:
logger.warning(
"Article chunk summary failed for section %d/%d of '%s'",
index + 1, total, title, exc_info=True,
)
# Fall back to the raw chunk truncated to ~1500 chars so the overall
# pipeline still delivers something rather than dropping the section.
return chunk[:1500]
async def prepare_article_context(
title: str,
url: str,
body: str,
model: str,
) -> str:
"""Return a conversation-ready context block for ``body``.
- Small article (≤ CHAR_BUDGET): returns ``body`` unchanged.
- Oversized article: runs a parallel map step over paragraph-aware
chunks and concatenates the summaries under section headers.
The returned string is what should go into the ``read_article`` synthetic
tool-result in chat history. Callers are responsible for caching it to
``rss_items.context_prepared``.
"""
body = body or ""
if len(body) <= CHAR_BUDGET:
return body
chunks = _chunk_by_paragraph(body)
logger.info(
"Article '%s' is %d chars, map-reducing into %d chunks",
title, len(body), len(chunks),
)
summaries = await asyncio.gather(
*[
_summarize_chunk(title, chunk, i, len(chunks), model)
for i, chunk in enumerate(chunks)
]
)
header = (
f"(This article was longer than the chat window could hold verbatim, "
f"so the full text was split into {len(chunks)} sections and each was "
"summarized below. Each section preserves specific claims, numbers, "
"and quotes from the original.)\n\n"
)
parts = [
f"## Section {i + 1}\n\n{summary}"
for i, summary in enumerate(summaries)
]
return header + "\n\n".join(parts)
@@ -1,12 +1,13 @@
"""Create and manage briefing conversations."""
import logging
from datetime import date, datetime, timezone
from datetime import datetime, timezone
from sqlalchemy import select
from fabledassistant.models import async_session
from fabledassistant.models.conversation import Conversation, Message
from fabledassistant.services.tz import user_briefing_date
logger = logging.getLogger(__name__)
@@ -15,7 +16,10 @@ async def get_or_create_today_conversation(user_id: int, model: str) -> Conversa
"""
Return today's briefing conversation, creating it if it doesn't exist.
"""
today = date.today()
# "Today" is the user-local briefing day (flips at 4am local), not
# ``date.today()`` — in a UTC container the latter rolls over at
# 19:00 NY local and makes the in-progress briefing disappear.
today = await user_briefing_date(user_id)
async with async_session() as session:
result = await session.execute(
select(Conversation).where(
@@ -445,9 +445,12 @@ async def _run_profile_closeout(user_id: int, model: str) -> None:
"""
from fabledassistant.services.user_profile import append_observations
from fabledassistant.services.llm import generate_completion
from fabledassistant.services.tz import user_today
from fabledassistant.models.conversation import Conversation, Message
yesterday = (date.today() - timedelta(days=1))
# User-local "yesterday" so closeout always targets the day that just
# ended in the user's timezone, regardless of container TZ.
yesterday = (await user_today(user_id)) - timedelta(days=1)
async with async_session() as session:
result = await session.execute(
select(Conversation).where(
@@ -527,9 +527,22 @@ async def run_generation(
should_gen_title = not conv_title or (msg_count > 0 and msg_count % 10 == 0)
if should_gen_title:
title_messages = messages + [
{"role": "assistant", "content": buf.content_so_far}
# Feed the title model the *raw* conversation turns only — never
# the post-build_context ``messages`` list. ``build_context``
# prepends RAG snippets, RSS excerpts, URL content, and briefing
# article dumps INTO the user message string itself, so filtering
# by role="user" downstream still surfaces that noise as the
# "user's message". That pollution caused wildly-wrong titles
# (bug #109) — the small background model was staring at article
# excerpts instead of what the user actually typed. Pass the
# original history + the raw user_content + the assistant reply.
title_messages: list[dict] = [
{"role": m["role"], "content": m.get("content") or ""}
for m in history
if m.get("role") in ("user", "assistant")
]
title_messages.append({"role": "user", "content": user_content})
title_messages.append({"role": "assistant", "content": buf.content_so_far})
async def _bg_title() -> None:
try:
+11 -3
View File
@@ -317,9 +317,17 @@ async def generate_completion(
num_ctx overrides the model's context window for this call only.
"""
last_exc: Exception | None = None
options: dict = {"num_predict": max_tokens}
if num_ctx is not None:
options["num_ctx"] = num_ctx
# Default num_ctx to Config.OLLAMA_NUM_CTX (matching stream_chat /
# stream_chat_with_tools). Without this, Ollama silently uses the model's
# default window (~4k on qwen3) and truncates anything longer. That is
# how the research pipeline's outline step kept falling back to a single
# monolith note: its 12-source prompt is ~6k tokens and was being chopped
# before the model ever saw it. Non-streaming callers must not inherit
# that footgun — if you truly want the model default, pass num_ctx=0.
options: dict = {
"num_predict": max_tokens,
"num_ctx": num_ctx if num_ctx is not None else Config.OLLAMA_NUM_CTX,
}
for attempt in range(3):
if attempt > 0:
delay = 3.0 * attempt
+18 -2
View File
@@ -63,7 +63,17 @@ async def _generate_outline(topic: str, sources: list[dict], model: str) -> list
]
for attempt in range(2):
try:
raw = await generate_completion(messages, model, max_tokens=400)
# Pin num_ctx explicitly. The prompt carries up to 12 sources at
# 2000 chars each (~6k tokens of source material alone) plus the
# system prompt — well over Ollama's default model window on
# qwen3. Without this, Ollama silently truncates the prompt, the
# model can't see most of the sources, JSON parsing fails twice,
# and the pipeline falls back to a single monolith note
# (`research.py:251`). Do not remove even if `generate_completion`
# appears to default this — see the comment there.
raw = await generate_completion(
messages, model, max_tokens=400, num_ctx=16384
)
raw = raw.strip()
raw = re.sub(r"^```(?:json)?\s*", "", raw)
raw = re.sub(r"\s*```$", "", raw)
@@ -161,7 +171,13 @@ async def _generate_executive_summary(
},
]
try:
raw = await generate_completion(messages, model, max_tokens=600)
# Pin num_ctx explicitly — see `_generate_outline` comment for the
# rationale. This prompt carries N sections × 1500 chars of section
# prose, which can easily exceed the default model window. Don't
# trust the `generate_completion` default to stick.
raw = await generate_completion(
messages, model, max_tokens=600, num_ctx=16384
)
return raw.strip()
except Exception:
logger.warning("Executive summary generation failed for '%s'", topic, exc_info=True)
+33
View File
@@ -34,6 +34,34 @@ def _html_to_text(html: str) -> str:
return html
async def get_or_fetch_full_article(item: RssItem) -> str | None:
"""Return the full article body, fetching+caching on miss.
Checks ``item.content_full`` first — populated either by the enrichment
pass at feed-ingest time or by a previous discuss-click. On miss, fetches
via ``_fetch_full_article`` and writes through. Returns ``None`` only if
the fetch itself fails; ``item.content_full == ""`` is still a cache hit.
Callers must pass an RssItem attached to an open session if they want
the write-through to persist — otherwise the fetched text is returned
but the cache stays empty and the next click will re-fetch.
"""
if item.content_full is not None:
return item.content_full
if not item.url:
return None
text = await _fetch_full_article(item.url)
if text is None:
return None
async with async_session() as session:
fresh = await session.get(RssItem, item.id)
if fresh is not None:
fresh.content_full = text
fresh.content_fetched_at = datetime.now(timezone.utc)
await session.commit()
return text
async def _fetch_full_article(url: str) -> str | None:
"""Fetch a URL and extract its main article text via trafilatura.
@@ -209,6 +237,11 @@ async def fetch_and_cache_feed(feed_id: int, url: str) -> int:
item = await session.get(RssItem, item_id)
if item:
item.content = full_text
# Populate the discuss-click cache too so the
# first click skips straight to the map-reduce
# step without re-fetching.
item.content_full = full_text
item.content_fetched_at = datetime.now(timezone.utc)
await session.commit()
await upsert_rss_item_embedding(
item_id, feed_user_id, item.title or "", item.content
+51 -26
View File
@@ -14,15 +14,37 @@ from fabledassistant.services.events import (
)
from fabledassistant.services.tools._helpers import resolve_project
from fabledassistant.services.tools._registry import tool
from fabledassistant.services.tz import get_user_tz
async def _parse_datetime_in_user_tz(
user_id: int, value: str
) -> tuple[datetime, bool]:
"""Parse a date/datetime string from the model into a UTC-aware datetime.
Naive inputs are interpreted in the **user's local timezone** and then
converted to UTC for storage. Never default to UTC for naive inputs —
that's how all-day events landed on the wrong day for non-UTC users.
Returns ``(utc_datetime, was_date_only)``.
"""
was_date_only = "T" not in value and " " not in value
if was_date_only:
value = f"{value}T00:00:00"
dt = datetime.fromisoformat(value.replace("Z", "+00:00"))
if dt.tzinfo is None:
user_tz = await get_user_tz(user_id)
dt = dt.replace(tzinfo=user_tz)
return dt.astimezone(timezone.utc), was_date_only
@tool(
name="create_event",
description="Create a calendar event for the user. Use this when the user asks to schedule, add, or create a meeting, appointment, or event.",
description="Create a calendar event for the user. Use this when the user asks to schedule, add, or create a meeting, appointment, or event. Pass dates and datetimes in the user's local time — a bare date like '2026-09-30' is interpreted as that day in the user's configured timezone.",
parameters={
"title": {"type": "string", "description": "A descriptive event title"},
"start": {"type": "string", "description": "Start date (YYYY-MM-DD) or datetime (ISO 8601 with timezone offset)"},
"end": {"type": "string", "description": "Optional end date or datetime"},
"start": {"type": "string", "description": "Start date (YYYY-MM-DD) or datetime (YYYY-MM-DDTHH:MM) in the user's local time. Include a timezone offset only if the user explicitly mentions one."},
"end": {"type": "string", "description": "Optional end date or datetime in the user's local time"},
"duration": {"type": "integer", "description": "Optional duration in minutes (default 60, ignored if end is set or all_day is true)"},
"description": {"type": "string", "description": "Optional event description"},
"location": {"type": "string", "description": "Optional event location"},
@@ -40,23 +62,21 @@ async def create_event_tool(*, user_id, arguments, **_ctx):
start_str = arguments["start"]
end_str = arguments.get("end")
all_day = arguments.get("all_day", False)
if "T" not in start_str and " " not in start_str:
all_day = True
start_str = f"{start_str}T00:00:00"
try:
start_dt = datetime.fromisoformat(start_str)
if start_dt.tzinfo is None:
start_dt = start_dt.replace(tzinfo=timezone.utc)
# Naive dates/datetimes are interpreted in the user's local
# timezone, not UTC. Storing UTC for naive inputs caused all-day
# events to land on the previous day for negative-offset users.
start_dt, start_was_date_only = await _parse_datetime_in_user_tz(
user_id, start_str
)
except (ValueError, TypeError):
return {"success": False, "error": f"Invalid start datetime: {start_str!r}"}
if start_was_date_only:
all_day = True
end_dt = None
if end_str:
if "T" not in end_str and " " not in end_str:
end_str = f"{end_str}T00:00:00"
try:
end_dt = datetime.fromisoformat(end_str)
if end_dt.tzinfo is None:
end_dt = end_dt.replace(tzinfo=timezone.utc)
end_dt, _ = await _parse_datetime_in_user_tz(user_id, end_str)
except (ValueError, TypeError):
return {"success": False, "error": f"Invalid end datetime: {end_str!r}"}
project_id = None
@@ -86,25 +106,30 @@ async def create_event_tool(*, user_id, arguments, **_ctx):
@tool(
name="list_events",
description="List calendar events in a date range. Use this when the user asks what events or meetings they have. Always use full-day UTC ranges: date_from at T00:00:00Z and date_to at T23:59:59Z for the days of interest so events stored in UTC are not missed.",
description="List calendar events in a date range. Use this when the user asks what events or meetings they have. Pass plain local dates (YYYY-MM-DD) — the server interprets them in the user's timezone and expands to a full local day.",
parameters={
"date_from": {"type": "string", "description": "Start of range in ISO 8601 UTC format (e.g. 2025-01-15T00:00:00Z). Use T00:00:00Z for the start of the day."},
"date_to": {"type": "string", "description": "End of range in ISO 8601 UTC format (e.g. 2025-01-22T23:59:59Z). Use T23:59:59Z for the end of the day."},
"date_from": {"type": "string", "description": "Start of range as a local date (YYYY-MM-DD) or local datetime. Interpreted in the user's timezone."},
"date_to": {"type": "string", "description": "End of range as a local date (YYYY-MM-DD) or local datetime. A bare date is expanded to 23:59:59 local."},
},
required=["date_from", "date_to"],
read_only=True,
briefing=True,
)
async def list_events_tool(*, user_id, arguments, **_ctx):
# Bare local dates are expanded to a full local day in the user's TZ:
# date_from → 00:00 local, date_to → 23:59:59 local, both converted to
# UTC before the DB query. Previously the tool description told the
# model to pass UTC ranges, which missed events for non-UTC users.
try:
date_from = datetime.fromisoformat(arguments["date_from"].replace("Z", "+00:00"))
date_to = datetime.fromisoformat(arguments["date_to"].replace("Z", "+00:00"))
date_from, _ = await _parse_datetime_in_user_tz(
user_id, arguments["date_from"]
)
date_to_str = arguments["date_to"]
if "T" not in date_to_str and " " not in date_to_str:
date_to_str = f"{date_to_str}T23:59:59"
date_to, _ = await _parse_datetime_in_user_tz(user_id, date_to_str)
except (ValueError, TypeError, KeyError) as exc:
return {"success": False, "error": f"Invalid date range: {exc}"}
if date_from.tzinfo is None:
date_from = date_from.replace(tzinfo=timezone.utc)
if date_to.tzinfo is None:
date_to = date_to.replace(tzinfo=timezone.utc)
events = await events_list_events(user_id=user_id, date_from=date_from, date_to=date_to)
return {
"success": True,
@@ -177,9 +202,9 @@ async def update_event_tool(*, user_id, arguments, **_ctx):
val = arguments.get(key)
if val:
try:
dt = datetime.fromisoformat(val.replace("Z", "+00:00"))
if dt.tzinfo is None:
dt = dt.replace(tzinfo=timezone.utc)
# Naive datetimes are user-local, not UTC — see
# ``_parse_datetime_in_user_tz`` docstring.
dt, _ = await _parse_datetime_in_user_tz(user_id, val)
fields[dt_field] = dt
except (ValueError, TypeError):
return {"success": False, "error": f"Invalid datetime for {key}: {val!r}"}
+47
View File
@@ -0,0 +1,47 @@
"""User-timezone helpers.
All datetimes in the DB are stored as UTC. The helpers here bridge between
that UTC storage and the user's configured local timezone (IANA string in
the ``user_timezone`` setting). Use these anywhere the model or UI talks
in terms of "today", "tomorrow", or a bare calendar date — never
``date.today()`` or ``datetime.now(timezone.utc)`` inside a per-user flow.
"""
from __future__ import annotations
from datetime import date, datetime, timedelta
from zoneinfo import ZoneInfo, ZoneInfoNotFoundError
from fabledassistant.services.settings import get_setting
# Briefing day boundary — kept in sync with the compilation slot in
# ``briefing_scheduler.SLOTS``. The briefing day flips at this local hour
# (not midnight) so the 00:0004:00 local window still shows yesterday's
# briefing until the 4am compilation generates the new one.
BRIEFING_DAY_START_HOUR = 4
async def get_user_tz(user_id: int) -> ZoneInfo:
"""Return the user's IANA ``ZoneInfo``, falling back to UTC."""
tz_str = await get_setting(user_id, "user_timezone") or "UTC"
try:
return ZoneInfo(tz_str)
except (ZoneInfoNotFoundError, KeyError):
return ZoneInfo("UTC")
async def user_today(user_id: int) -> date:
"""Return today's calendar date in the user's local timezone."""
tz = await get_user_tz(user_id)
return datetime.now(tz).date()
async def user_briefing_date(user_id: int) -> date:
"""Return the current "briefing day" in the user's local timezone.
The briefing day flips at ``BRIEFING_DAY_START_HOUR`` (4am local),
aligned with the compilation slot that generates the day's briefing.
Between 00:00 and 04:00 local this still returns *yesterday*, so the
UI keeps showing the in-progress briefing until the new one is built.
"""
tz = await get_user_tz(user_id)
return (datetime.now(tz) - timedelta(hours=BRIEFING_DAY_START_HOUR)).date()
+80
View File
@@ -134,3 +134,83 @@ def test_history_builder_no_tool_calls_unchanged():
assert len(history) == 2
assert history[0] == {"role": "user", "content": "Hello"}
assert history[1] == {"role": "assistant", "content": "Hi there!"}
# ---------------------------------------------------------------------------
# prepare_article_context tests
# ---------------------------------------------------------------------------
@pytest.mark.asyncio
async def test_prepare_article_context_small_passthrough():
"""Articles under CHAR_BUDGET pass through unchanged with zero LLM calls."""
from fabledassistant.services import article_context
body = "A short article.\n\nWith two paragraphs."
with patch(
"fabledassistant.services.article_context.generate_completion",
new_callable=AsyncMock,
) as mock_gen:
out = await article_context.prepare_article_context(
"Title", "https://example.com", body, "test-model",
)
assert out == body
mock_gen.assert_not_called()
@pytest.mark.asyncio
async def test_prepare_article_context_large_runs_map_reduce():
"""Articles over CHAR_BUDGET are chunked and map-reduced via the background model."""
from fabledassistant.services import article_context
# CHAR_BUDGET is 48_000 — build a body well over that with paragraph breaks
# so the chunker has natural splits to work with.
paragraph = ("Lorem ipsum dolor sit amet, consectetur adipiscing elit. " * 40).strip()
body = "\n\n".join([paragraph] * 30) # ~70k+ chars across 30 paragraphs
assert len(body) > article_context.CHAR_BUDGET
with patch(
"fabledassistant.services.article_context.generate_completion",
new_callable=AsyncMock,
return_value="Summary of this section with specific claims preserved.",
) as mock_gen:
out = await article_context.prepare_article_context(
"Long Article", "https://example.com/long", body, "test-model",
)
# At least one LLM call fired (the map step ran)
assert mock_gen.await_count >= 1
# Output carries the oversized-article header and section markers
assert "longer than the chat window" in out
assert "## Section 1" in out
# Map output is much smaller than the raw body
assert len(out) < len(body)
def test_chunk_by_paragraph_respects_boundaries():
"""Chunker splits on paragraph breaks, not mid-sentence."""
from fabledassistant.services.article_context import _chunk_by_paragraph, CHUNK_CHARS
paragraphs = [f"Paragraph {i}. " + ("x" * 1000) for i in range(20)]
body = "\n\n".join(paragraphs)
chunks = _chunk_by_paragraph(body)
# Each chunk stays under the budget
for chunk in chunks:
assert len(chunk) <= CHUNK_CHARS
# Total content is preserved (modulo overlap duplication, so ≥ original)
assert sum(len(c) for c in chunks) >= len(body) * 0.9
def test_chunk_by_paragraph_handles_oversized_paragraph():
"""A single paragraph larger than CHUNK_CHARS gets split mid-paragraph."""
from fabledassistant.services.article_context import _chunk_by_paragraph, CHUNK_CHARS
body = "x" * (CHUNK_CHARS * 3) # one huge paragraph, no breaks
chunks = _chunk_by_paragraph(body)
assert len(chunks) >= 3
for chunk in chunks:
assert len(chunk) <= CHUNK_CHARS
+134
View File
@@ -0,0 +1,134 @@
"""Regression tests: calendar tool respects the user's configured timezone."""
from datetime import timezone
from unittest.mock import AsyncMock, patch
import pytest
@pytest.mark.asyncio
async def test_create_event_all_day_bare_date_interprets_local_midnight():
"""Bare date like '2026-09-30' for a NY user = 2026-09-30 00:00 NY,
stored as 2026-09-30 04:00 UTC — NOT 2026-09-30 00:00 UTC (which would
be 2026-09-29 19:00 NY and land on the wrong day)."""
from fabledassistant.services.tools.calendar import create_event_tool
captured = {}
async def fake_create_event(**kwargs):
captured.update(kwargs)
event = AsyncMock()
event.to_dict.return_value = {"id": 1, **{k: v for k, v in kwargs.items() if not callable(v)}}
return event
with patch(
"fabledassistant.services.tools.calendar.get_user_tz",
AsyncMock(return_value=__import__("zoneinfo").ZoneInfo("America/New_York")),
), patch(
"fabledassistant.services.tools.calendar.events_create_event",
side_effect=fake_create_event,
):
result = await create_event_tool(
user_id=1,
arguments={"title": "Birthday", "start": "2026-09-30"},
)
assert result["success"] is True
start_dt = captured["start_dt"]
assert start_dt.tzinfo is not None
# Converted to UTC: 00:00 NY (EDT, UTC-4) == 04:00 UTC on the same day
utc = start_dt.astimezone(timezone.utc)
assert (utc.year, utc.month, utc.day) == (2026, 9, 30)
assert utc.hour == 4 # EDT offset; would be 5 in EST
assert captured["all_day"] is True
@pytest.mark.asyncio
async def test_create_event_naive_datetime_is_user_local():
"""'2026-03-15T14:30' for a NY user means 2:30pm NY, not 2:30pm UTC."""
from fabledassistant.services.tools.calendar import create_event_tool
captured = {}
async def fake_create_event(**kwargs):
captured.update(kwargs)
event = AsyncMock()
event.to_dict.return_value = {"id": 1}
return event
with patch(
"fabledassistant.services.tools.calendar.get_user_tz",
AsyncMock(return_value=__import__("zoneinfo").ZoneInfo("America/New_York")),
), patch(
"fabledassistant.services.tools.calendar.events_create_event",
side_effect=fake_create_event,
):
await create_event_tool(
user_id=1,
arguments={"title": "Meeting", "start": "2026-03-15T14:30"},
)
utc = captured["start_dt"].astimezone(timezone.utc)
# 14:30 NY on 2026-03-15 (after DST start) = 18:30 UTC
assert (utc.year, utc.month, utc.day, utc.hour, utc.minute) == (2026, 3, 15, 18, 30)
@pytest.mark.asyncio
async def test_create_event_explicit_offset_is_respected():
"""Model-supplied timezone offsets must not be overridden by user TZ."""
from fabledassistant.services.tools.calendar import create_event_tool
captured = {}
async def fake_create_event(**kwargs):
captured.update(kwargs)
event = AsyncMock()
event.to_dict.return_value = {"id": 1}
return event
with patch(
"fabledassistant.services.tools.calendar.get_user_tz",
AsyncMock(return_value=__import__("zoneinfo").ZoneInfo("America/New_York")),
), patch(
"fabledassistant.services.tools.calendar.events_create_event",
side_effect=fake_create_event,
):
await create_event_tool(
user_id=1,
arguments={"title": "Meeting", "start": "2026-03-15T14:30+00:00"},
)
utc = captured["start_dt"].astimezone(timezone.utc)
assert (utc.hour, utc.minute) == (14, 30)
@pytest.mark.asyncio
async def test_list_events_bare_date_range_covers_local_day():
"""date_from/date_to as bare dates should cover the full LOCAL day."""
from fabledassistant.services.tools.calendar import list_events_tool
captured = {}
async def fake_list_events(*, user_id, date_from, date_to):
captured["date_from"] = date_from
captured["date_to"] = date_to
return []
with patch(
"fabledassistant.services.tools.calendar.get_user_tz",
AsyncMock(return_value=__import__("zoneinfo").ZoneInfo("America/New_York")),
), patch(
"fabledassistant.services.tools.calendar.events_list_events",
side_effect=fake_list_events,
):
await list_events_tool(
user_id=1,
arguments={"date_from": "2026-09-30", "date_to": "2026-09-30"},
)
df = captured["date_from"].astimezone(timezone.utc)
dt = captured["date_to"].astimezone(timezone.utc)
# 2026-09-30 00:00 NY (EDT) = 04:00 UTC same day
assert (df.year, df.month, df.day, df.hour) == (2026, 9, 30, 4)
# 2026-09-30 23:59:59 NY (EDT) = 03:59:59 UTC next day
assert (dt.year, dt.month, dt.day, dt.hour) == (2026, 10, 1, 3)
+63
View File
@@ -0,0 +1,63 @@
"""Tests for user-timezone helpers (services/tz.py)."""
from datetime import datetime
from unittest.mock import AsyncMock, patch
from zoneinfo import ZoneInfo
import pytest
@pytest.mark.asyncio
async def test_get_user_tz_returns_configured_zone():
from fabledassistant.services import tz as tz_mod
with patch.object(tz_mod, "get_setting", AsyncMock(return_value="America/New_York")):
zone = await tz_mod.get_user_tz(1)
assert zone == ZoneInfo("America/New_York")
@pytest.mark.asyncio
async def test_get_user_tz_falls_back_to_utc_on_bad_input():
from fabledassistant.services import tz as tz_mod
with patch.object(tz_mod, "get_setting", AsyncMock(return_value="Not/AZone")):
zone = await tz_mod.get_user_tz(1)
assert zone == ZoneInfo("UTC")
@pytest.mark.asyncio
async def test_user_briefing_date_before_4am_returns_yesterday():
"""00:0003:59 local still shows yesterday's briefing day."""
from fabledassistant.services import tz as tz_mod
ny = ZoneInfo("America/New_York")
# 02:00 NY on 2026-04-13 → briefing day = 2026-04-12
fake_now = datetime(2026, 4, 13, 2, 0, tzinfo=ny)
class _FakeDatetime(datetime):
@classmethod
def now(cls, tz=None):
return fake_now
with patch.object(tz_mod, "get_user_tz", AsyncMock(return_value=ny)), \
patch.object(tz_mod, "datetime", _FakeDatetime):
day = await tz_mod.user_briefing_date(1)
assert day.isoformat() == "2026-04-12"
@pytest.mark.asyncio
async def test_user_briefing_date_after_4am_returns_today():
from fabledassistant.services import tz as tz_mod
ny = ZoneInfo("America/New_York")
fake_now = datetime(2026, 4, 13, 4, 30, tzinfo=ny)
class _FakeDatetime(datetime):
@classmethod
def now(cls, tz=None):
return fake_now
with patch.object(tz_mod, "get_user_tz", AsyncMock(return_value=ny)), \
patch.object(tz_mod, "datetime", _FakeDatetime):
day = await tz_mod.user_briefing_date(1)
assert day.isoformat() == "2026-04-13"