Compare commits

..

6 Commits

Author SHA1 Message Date
bvandeusen fd885c1bc8 Merge pull request 'Release v26.04.13.3' (#31) from dev into main 2026-04-14 03:36:09 +00:00
bvandeusen 8205590f8d feat(briefing): cache + map-reduce article context for rich discuss chats
The Discuss button on news cards was producing one-shot replies because
the model got the whole trafilatura blob dropped into history with a
canned "summarize and discuss this article" prompt — no length guard, no
prep, no invitation to converse. Large articles got silently truncated by
Ollama; small articles got a tepid reply.

This reworks discuss_article around a three-layer cache:

  context_prepared  →  content_full  →  fresh trafilatura fetch

First click on a small article fetches once, writes through to both
caches, and passes the body straight into the synthetic read_article
tool-result. First click on a large article additionally runs a parallel
map step (services/article_context.py) that chunks the body on paragraph
boundaries, summarizes each ~8k chunk to ~300 words of dense factual
prose via the background model, and concatenates the summaries under
section headers — all pinned to num_ctx=16384 so the map step doesn't
itself fall victim to silent truncation. Repeat clicks on either path
skip straight to the chat turn.

The canned summary prompt is replaced with a conversational seed that
invites the user into an actual discussion rather than a one-shot
synopsis, matching the goal of "have a conversation about an article,
not just read it."

discuss_topic is intentionally left untouched — it's the multi-article
aggregation path and needs a separate rework. Follow-up task will decide
whether to retire it or rework it on the cached-context approach.

Closes task #106.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-13 20:52:00 -04:00
bvandeusen 939b910372 Merge pull request 'Release v26.04.13.2' (#30) from dev into main 2026-04-13 23:00:33 +00:00
bvandeusen 70cea78c2f fix(llm): default generate_completion num_ctx to Config.OLLAMA_NUM_CTX
Non-streaming generate_completion was the only LLM entry point that
didn't default num_ctx — stream_chat and stream_chat_with_tools both
fall back to Config.OLLAMA_NUM_CTX (16384). When a caller omitted the
argument, Ollama silently used the model's default window (~4k on
qwen3) and truncated the prompt.

That footgun was masked by fallback paths in the research pipeline:
_generate_outline's prompt carries ~12 sources × 2000 chars (~6k
tokens) of source material plus a system prompt, so the prompt got
chopped, the model never saw the sources, JSON parsing failed twice,
and run_research_pipeline dropped into the single-note "monolith"
fallback (research.py:251). The user reported chat 215 producing such
a monolith note for a multi-source research topic.

Two-layer fix:
- Default num_ctx to Config.OLLAMA_NUM_CTX inside generate_completion,
  matching the streaming entry points. Any current or future caller
  that forgets the argument stops silently losing input.
- Pin num_ctx=16384 explicitly in _generate_outline and
  _generate_executive_summary with comments pointing at the failure
  mode, so a refactor of the generate_completion default can't
  silently regress the research pipeline.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-13 18:20:58 -04:00
bvandeusen 4e4dbb8783 fix(chat): feed title model raw turns instead of post-build_context messages
_generate_title was receiving the full messages list from build_context,
which prepends RAG snippets, RSS excerpts, URL content, and briefing
article dumps INTO the user-role message string. The role=="user" filter
inside _generate_title then handed that composite blob (capped at 300
chars) to gemma3:4b as "the user's message", so the background model
was titling conversations based on article excerpts instead of what the
user actually typed — producing wildly wrong titles like "Briefing
Profile Preferences & Schedule" for a plain calendar query. See #109.

Pass the raw history + user_content + assistant reply instead.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-13 17:15:39 -04:00
bvandeusen e4e1d1da49 fix(tz): interpret calendar and briefing dates in user's local timezone
Two related bugs where the server defaulted naive datetimes to UTC instead
of the configured user timezone, causing all-day events to land on the
previous day and briefings to "disappear" at UTC midnight.

- New services/tz.py helpers: get_user_tz, user_today, user_briefing_date
  (the briefing day flips at 4am local to align with the compilation slot,
  so the 00:00-04:00 local window still shows yesterday's briefing until
  the new one is generated).
- calendar create/list/update tools now parse naive datetimes in the
  user's TZ before converting to UTC for storage, and tool descriptions
  tell the model to pass plain local dates.
- briefing_conversations.get_or_create_today_conversation and the
  reset-today route use user_briefing_date so the in-progress briefing
  doesn't get replaced at 19:00 NY / UTC midnight.
- _run_profile_closeout targets user-local "yesterday" for consistency.

Regression tests added for the TZ helpers and the calendar tool.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-13 15:35:27 -04:00
15 changed files with 707 additions and 45 deletions
@@ -0,0 +1,34 @@
"""Add content_full and context_prepared caches to rss_items.
Revision ID: 0038
Revises: 0037
Create Date: 2026-04-13
"""
from __future__ import annotations
import sqlalchemy as sa
from alembic import op
revision = "0038"
down_revision = "0037"
branch_labels = None
depends_on = None
def upgrade() -> None:
op.add_column("rss_items", sa.Column("content_full", sa.Text(), nullable=True))
op.add_column(
"rss_items",
sa.Column("context_prepared", sa.Text(), nullable=True),
)
op.add_column(
"rss_items",
sa.Column("content_fetched_at", sa.DateTime(timezone=True), nullable=True),
)
def downgrade() -> None:
op.drop_column("rss_items", "content_fetched_at")
op.drop_column("rss_items", "context_prepared")
op.drop_column("rss_items", "content_full")
+11
View File
@@ -46,6 +46,17 @@ class RssItem(Base):
published_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True) published_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True)
# Truncated to 2000 chars to keep DB size reasonable # Truncated to 2000 chars to keep DB size reasonable
content: Mapped[str] = mapped_column(Text, default="") content: Mapped[str] = mapped_column(Text, default="")
# Full trafilatura-extracted article body, populated lazily on first
# discuss-click / enrichment pass. Nullable — most items never get this
# cached. Expires naturally with the item (90-day retention).
content_full: Mapped[str | None] = mapped_column(Text, nullable=True)
# Map-reduced conversation-ready context derived from content_full. See
# services/article_context.py — populated on first discuss click so
# repeat clicks skip both the fetch and the LLM map step.
context_prepared: Mapped[str | None] = mapped_column(Text, nullable=True)
content_fetched_at: Mapped[datetime | None] = mapped_column(
DateTime(timezone=True), nullable=True
)
fetched_at: Mapped[datetime] = mapped_column( fetched_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True), default=lambda: datetime.now(timezone.utc) DateTime(timezone=True), default=lambda: datetime.now(timezone.utc)
) )
+39 -9
View File
@@ -330,10 +330,13 @@ async def reset_today_briefing():
driven from the MCP when iterating on prompts. Pair with driven from the MCP when iterating on prompts. Pair with
``POST /api/briefing/trigger`` to immediately regenerate. ``POST /api/briefing/trigger`` to immediately regenerate.
""" """
from datetime import date as _date
from sqlalchemy import delete as _delete from sqlalchemy import delete as _delete
today = _date.today() from fabledassistant.services.tz import user_briefing_date
# User-local briefing day (flips at 4am local), not ``date.today()`` —
# see services/tz.py for rationale.
today = await user_briefing_date(g.user.id)
async with async_session() as session: async with async_session() as session:
result = await session.execute( result = await session.execute(
select(Conversation).where( select(Conversation).where(
@@ -529,8 +532,28 @@ async def discuss_article(item_id: int):
if get_buffer(conv_id) is not None: if get_buffer(conv_id) is not None:
return jsonify({"error": "Generation already in progress"}), 409 return jsonify({"error": "Generation already in progress"}), 409
from fabledassistant.services.rss import _fetch_full_article # Three-layer cache: context_prepared (post-map-reduce) → content_full
article_content = await _fetch_full_article(item.url) or item.content or "" # (raw trafilatura) → fresh fetch. Only the first miss pays the fetch
# cost; only a large uncached article pays the map-reduce cost. Repeat
# clicks on the same article skip straight to the chat turn.
from fabledassistant.services.article_context import prepare_article_context
from fabledassistant.services.rss import get_or_fetch_full_article
model = await get_setting(uid, "default_model", "") or ""
if item.context_prepared:
article_content = item.context_prepared
else:
raw_body = await get_or_fetch_full_article(item) or item.content or ""
article_content = await prepare_article_context(
item.title or "", item.url, raw_body, model,
)
if article_content:
async with async_session() as session:
fresh = await session.get(RssItem, item.id)
if fresh is not None:
fresh.context_prepared = article_content
await session.commit()
# Store synthetic assistant message with read_article tool result # Store synthetic assistant message with read_article tool result
synthetic_tool_calls = [{ synthetic_tool_calls = [{
@@ -546,8 +569,17 @@ async def discuss_article(item_id: int):
}] }]
await add_message(conv_id, "assistant", "", status="complete", tool_calls=synthetic_tool_calls) await add_message(conv_id, "assistant", "", status="complete", tool_calls=synthetic_tool_calls)
# Store user message # Conversational seed — invites a real discussion rather than asking for
await add_message(conv_id, "user", "Please summarize and discuss this article.") # a one-shot summary. The model sees the article context in the tool
# result above and responds to this user turn as the start of an ongoing
# conversation the user will steer with follow-ups.
discuss_prompt = (
"I want to talk about this article. Start with a substantive summary "
"of what it's arguing and the key evidence it uses, then tell me what "
"stood out to you or seems worth pushing back on. I'll ask follow-ups "
"from there."
)
await add_message(conv_id, "user", discuss_prompt)
# Reload conversation with fresh messages to build history # Reload conversation with fresh messages to build history
conv = await get_conversation(uid, conv_id) conv = await get_conversation(uid, conv_id)
@@ -569,15 +601,13 @@ async def discuss_article(item_id: int):
else: else:
history.append(msg_dict) history.append(msg_dict)
model = await get_setting(uid, "default_model", "") or ""
assistant_msg = await add_message(conv_id, "assistant", "", status="generating") assistant_msg = await add_message(conv_id, "assistant", "", status="generating")
buf = create_buffer(conv_id, assistant_msg.id) buf = create_buffer(conv_id, assistant_msg.id)
asyncio.create_task(run_generation( asyncio.create_task(run_generation(
buf, history, model, buf, history, model,
uid, conv_id, conv.title or "", uid, conv_id, conv.title or "",
"Please summarize and discuss this article.", discuss_prompt,
)) ))
return jsonify({"assistant_message_id": assistant_msg.id, "status": "generating"}), 202 return jsonify({"assistant_message_id": assistant_msg.id, "status": "generating"}), 202
@@ -0,0 +1,161 @@
"""Prepare article bodies as conversation-ready context.
Used by the briefing ``discuss-article`` flow. A raw trafilatura extraction
is often too large to drop whole into a chat history without eating the
context window, so this module runs a map-reduce step over oversized
articles and returns a compact, structured context that still preserves the
article's meaning across sections.
Small articles pass through unchanged — map-reduce only fires when the raw
body exceeds CHAR_BUDGET. The output is cached on ``rss_items.context_prepared``
by the caller, so repeat discuss-clicks on the same article skip this work
entirely.
"""
from __future__ import annotations
import asyncio
import logging
import re
from fabledassistant.services.llm import generate_completion
logger = logging.getLogger(__name__)
# ~12k tokens at 4 chars/token. Comfortably under OLLAMA_NUM_CTX=16384
# with room left for system prompt, chat history, and the assistant reply.
CHAR_BUDGET = 48_000
# Chunk size for the map step on oversized articles. Overlap preserves
# context across paragraph boundaries that happen to land mid-sentence.
CHUNK_CHARS = 8_000
CHUNK_OVERLAP = 400
_PARA_SPLIT = re.compile(r"\n\s*\n")
def _chunk_by_paragraph(body: str) -> list[str]:
"""Split ``body`` into chunks of up to CHUNK_CHARS, respecting paragraphs.
Paragraphs longer than CHUNK_CHARS are split mid-paragraph as a last
resort. Adjacent chunks share CHUNK_OVERLAP chars of trailing text so
a sentence straddling the boundary stays readable on both sides.
"""
paragraphs = [p.strip() for p in _PARA_SPLIT.split(body) if p.strip()]
chunks: list[str] = []
current: list[str] = []
current_len = 0
for para in paragraphs:
para_len = len(para)
if para_len > CHUNK_CHARS:
if current:
chunks.append("\n\n".join(current))
current, current_len = [], 0
for i in range(0, para_len, CHUNK_CHARS - CHUNK_OVERLAP):
chunks.append(para[i : i + CHUNK_CHARS])
continue
if current_len + para_len + 2 > CHUNK_CHARS and current:
chunks.append("\n\n".join(current))
tail = current[-1][-CHUNK_OVERLAP:] if current else ""
current = [tail, para] if tail else [para]
current_len = len(tail) + para_len + (2 if tail else 0)
else:
current.append(para)
current_len += para_len + 2
if current:
chunks.append("\n\n".join(current))
return chunks
async def _summarize_chunk(title: str, chunk: str, index: int, total: int, model: str) -> str:
"""Map-step summary of one article chunk.
Aims for ~300 words of dense, factual prose — not bullet points — so the
downstream chat model can quote from it naturally.
"""
messages = [
{
"role": "system",
"content": (
"You are summarizing one section of a larger article so a downstream "
"conversation model can discuss the full article without having to read "
"every word.\n\n"
"Requirements:\n"
"- 250350 words of dense factual prose\n"
"- Preserve specific claims, numbers, names, and quotes\n"
"- Do NOT editorialize or add analysis\n"
"- Do NOT use bullet points or headings\n"
"- Do NOT say 'this section' or 'this article' — write content, not meta"
),
},
{
"role": "user",
"content": (
f"Article: {title}\n"
f"Section {index + 1} of {total}:\n\n{chunk}"
),
},
]
try:
# Pin num_ctx — same rationale as services/research.py:66. A large
# chunk plus system prompt can push well past the default window;
# silent truncation here would drop the tail of the chunk without
# any error, producing a misleading summary.
raw = await generate_completion(
messages, model, max_tokens=600, num_ctx=16384
)
return raw.strip()
except Exception:
logger.warning(
"Article chunk summary failed for section %d/%d of '%s'",
index + 1, total, title, exc_info=True,
)
# Fall back to the raw chunk truncated to ~1500 chars so the overall
# pipeline still delivers something rather than dropping the section.
return chunk[:1500]
async def prepare_article_context(
title: str,
url: str,
body: str,
model: str,
) -> str:
"""Return a conversation-ready context block for ``body``.
- Small article (≤ CHAR_BUDGET): returns ``body`` unchanged.
- Oversized article: runs a parallel map step over paragraph-aware
chunks and concatenates the summaries under section headers.
The returned string is what should go into the ``read_article`` synthetic
tool-result in chat history. Callers are responsible for caching it to
``rss_items.context_prepared``.
"""
body = body or ""
if len(body) <= CHAR_BUDGET:
return body
chunks = _chunk_by_paragraph(body)
logger.info(
"Article '%s' is %d chars, map-reducing into %d chunks",
title, len(body), len(chunks),
)
summaries = await asyncio.gather(
*[
_summarize_chunk(title, chunk, i, len(chunks), model)
for i, chunk in enumerate(chunks)
]
)
header = (
f"(This article was longer than the chat window could hold verbatim, "
f"so the full text was split into {len(chunks)} sections and each was "
"summarized below. Each section preserves specific claims, numbers, "
"and quotes from the original.)\n\n"
)
parts = [
f"## Section {i + 1}\n\n{summary}"
for i, summary in enumerate(summaries)
]
return header + "\n\n".join(parts)
@@ -1,12 +1,13 @@
"""Create and manage briefing conversations.""" """Create and manage briefing conversations."""
import logging import logging
from datetime import date, datetime, timezone from datetime import datetime, timezone
from sqlalchemy import select from sqlalchemy import select
from fabledassistant.models import async_session from fabledassistant.models import async_session
from fabledassistant.models.conversation import Conversation, Message from fabledassistant.models.conversation import Conversation, Message
from fabledassistant.services.tz import user_briefing_date
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
@@ -15,7 +16,10 @@ async def get_or_create_today_conversation(user_id: int, model: str) -> Conversa
""" """
Return today's briefing conversation, creating it if it doesn't exist. Return today's briefing conversation, creating it if it doesn't exist.
""" """
today = date.today() # "Today" is the user-local briefing day (flips at 4am local), not
# ``date.today()`` — in a UTC container the latter rolls over at
# 19:00 NY local and makes the in-progress briefing disappear.
today = await user_briefing_date(user_id)
async with async_session() as session: async with async_session() as session:
result = await session.execute( result = await session.execute(
select(Conversation).where( select(Conversation).where(
@@ -445,9 +445,12 @@ async def _run_profile_closeout(user_id: int, model: str) -> None:
""" """
from fabledassistant.services.user_profile import append_observations from fabledassistant.services.user_profile import append_observations
from fabledassistant.services.llm import generate_completion from fabledassistant.services.llm import generate_completion
from fabledassistant.services.tz import user_today
from fabledassistant.models.conversation import Conversation, Message from fabledassistant.models.conversation import Conversation, Message
yesterday = (date.today() - timedelta(days=1)) # User-local "yesterday" so closeout always targets the day that just
# ended in the user's timezone, regardless of container TZ.
yesterday = (await user_today(user_id)) - timedelta(days=1)
async with async_session() as session: async with async_session() as session:
result = await session.execute( result = await session.execute(
select(Conversation).where( select(Conversation).where(
@@ -527,9 +527,22 @@ async def run_generation(
should_gen_title = not conv_title or (msg_count > 0 and msg_count % 10 == 0) should_gen_title = not conv_title or (msg_count > 0 and msg_count % 10 == 0)
if should_gen_title: if should_gen_title:
title_messages = messages + [ # Feed the title model the *raw* conversation turns only — never
{"role": "assistant", "content": buf.content_so_far} # the post-build_context ``messages`` list. ``build_context``
# prepends RAG snippets, RSS excerpts, URL content, and briefing
# article dumps INTO the user message string itself, so filtering
# by role="user" downstream still surfaces that noise as the
# "user's message". That pollution caused wildly-wrong titles
# (bug #109) — the small background model was staring at article
# excerpts instead of what the user actually typed. Pass the
# original history + the raw user_content + the assistant reply.
title_messages: list[dict] = [
{"role": m["role"], "content": m.get("content") or ""}
for m in history
if m.get("role") in ("user", "assistant")
] ]
title_messages.append({"role": "user", "content": user_content})
title_messages.append({"role": "assistant", "content": buf.content_so_far})
async def _bg_title() -> None: async def _bg_title() -> None:
try: try:
+11 -3
View File
@@ -317,9 +317,17 @@ async def generate_completion(
num_ctx overrides the model's context window for this call only. num_ctx overrides the model's context window for this call only.
""" """
last_exc: Exception | None = None last_exc: Exception | None = None
options: dict = {"num_predict": max_tokens} # Default num_ctx to Config.OLLAMA_NUM_CTX (matching stream_chat /
if num_ctx is not None: # stream_chat_with_tools). Without this, Ollama silently uses the model's
options["num_ctx"] = num_ctx # default window (~4k on qwen3) and truncates anything longer. That is
# how the research pipeline's outline step kept falling back to a single
# monolith note: its 12-source prompt is ~6k tokens and was being chopped
# before the model ever saw it. Non-streaming callers must not inherit
# that footgun — if you truly want the model default, pass num_ctx=0.
options: dict = {
"num_predict": max_tokens,
"num_ctx": num_ctx if num_ctx is not None else Config.OLLAMA_NUM_CTX,
}
for attempt in range(3): for attempt in range(3):
if attempt > 0: if attempt > 0:
delay = 3.0 * attempt delay = 3.0 * attempt
+18 -2
View File
@@ -63,7 +63,17 @@ async def _generate_outline(topic: str, sources: list[dict], model: str) -> list
] ]
for attempt in range(2): for attempt in range(2):
try: try:
raw = await generate_completion(messages, model, max_tokens=400) # Pin num_ctx explicitly. The prompt carries up to 12 sources at
# 2000 chars each (~6k tokens of source material alone) plus the
# system prompt — well over Ollama's default model window on
# qwen3. Without this, Ollama silently truncates the prompt, the
# model can't see most of the sources, JSON parsing fails twice,
# and the pipeline falls back to a single monolith note
# (`research.py:251`). Do not remove even if `generate_completion`
# appears to default this — see the comment there.
raw = await generate_completion(
messages, model, max_tokens=400, num_ctx=16384
)
raw = raw.strip() raw = raw.strip()
raw = re.sub(r"^```(?:json)?\s*", "", raw) raw = re.sub(r"^```(?:json)?\s*", "", raw)
raw = re.sub(r"\s*```$", "", raw) raw = re.sub(r"\s*```$", "", raw)
@@ -161,7 +171,13 @@ async def _generate_executive_summary(
}, },
] ]
try: try:
raw = await generate_completion(messages, model, max_tokens=600) # Pin num_ctx explicitly — see `_generate_outline` comment for the
# rationale. This prompt carries N sections × 1500 chars of section
# prose, which can easily exceed the default model window. Don't
# trust the `generate_completion` default to stick.
raw = await generate_completion(
messages, model, max_tokens=600, num_ctx=16384
)
return raw.strip() return raw.strip()
except Exception: except Exception:
logger.warning("Executive summary generation failed for '%s'", topic, exc_info=True) logger.warning("Executive summary generation failed for '%s'", topic, exc_info=True)
+33
View File
@@ -34,6 +34,34 @@ def _html_to_text(html: str) -> str:
return html return html
async def get_or_fetch_full_article(item: RssItem) -> str | None:
"""Return the full article body, fetching+caching on miss.
Checks ``item.content_full`` first — populated either by the enrichment
pass at feed-ingest time or by a previous discuss-click. On miss, fetches
via ``_fetch_full_article`` and writes through. Returns ``None`` only if
the fetch itself fails; ``item.content_full == ""`` is still a cache hit.
Callers must pass an RssItem attached to an open session if they want
the write-through to persist — otherwise the fetched text is returned
but the cache stays empty and the next click will re-fetch.
"""
if item.content_full is not None:
return item.content_full
if not item.url:
return None
text = await _fetch_full_article(item.url)
if text is None:
return None
async with async_session() as session:
fresh = await session.get(RssItem, item.id)
if fresh is not None:
fresh.content_full = text
fresh.content_fetched_at = datetime.now(timezone.utc)
await session.commit()
return text
async def _fetch_full_article(url: str) -> str | None: async def _fetch_full_article(url: str) -> str | None:
"""Fetch a URL and extract its main article text via trafilatura. """Fetch a URL and extract its main article text via trafilatura.
@@ -209,6 +237,11 @@ async def fetch_and_cache_feed(feed_id: int, url: str) -> int:
item = await session.get(RssItem, item_id) item = await session.get(RssItem, item_id)
if item: if item:
item.content = full_text item.content = full_text
# Populate the discuss-click cache too so the
# first click skips straight to the map-reduce
# step without re-fetching.
item.content_full = full_text
item.content_fetched_at = datetime.now(timezone.utc)
await session.commit() await session.commit()
await upsert_rss_item_embedding( await upsert_rss_item_embedding(
item_id, feed_user_id, item.title or "", item.content item_id, feed_user_id, item.title or "", item.content
+51 -26
View File
@@ -14,15 +14,37 @@ from fabledassistant.services.events import (
) )
from fabledassistant.services.tools._helpers import resolve_project from fabledassistant.services.tools._helpers import resolve_project
from fabledassistant.services.tools._registry import tool from fabledassistant.services.tools._registry import tool
from fabledassistant.services.tz import get_user_tz
async def _parse_datetime_in_user_tz(
user_id: int, value: str
) -> tuple[datetime, bool]:
"""Parse a date/datetime string from the model into a UTC-aware datetime.
Naive inputs are interpreted in the **user's local timezone** and then
converted to UTC for storage. Never default to UTC for naive inputs —
that's how all-day events landed on the wrong day for non-UTC users.
Returns ``(utc_datetime, was_date_only)``.
"""
was_date_only = "T" not in value and " " not in value
if was_date_only:
value = f"{value}T00:00:00"
dt = datetime.fromisoformat(value.replace("Z", "+00:00"))
if dt.tzinfo is None:
user_tz = await get_user_tz(user_id)
dt = dt.replace(tzinfo=user_tz)
return dt.astimezone(timezone.utc), was_date_only
@tool( @tool(
name="create_event", name="create_event",
description="Create a calendar event for the user. Use this when the user asks to schedule, add, or create a meeting, appointment, or event.", description="Create a calendar event for the user. Use this when the user asks to schedule, add, or create a meeting, appointment, or event. Pass dates and datetimes in the user's local time — a bare date like '2026-09-30' is interpreted as that day in the user's configured timezone.",
parameters={ parameters={
"title": {"type": "string", "description": "A descriptive event title"}, "title": {"type": "string", "description": "A descriptive event title"},
"start": {"type": "string", "description": "Start date (YYYY-MM-DD) or datetime (ISO 8601 with timezone offset)"}, "start": {"type": "string", "description": "Start date (YYYY-MM-DD) or datetime (YYYY-MM-DDTHH:MM) in the user's local time. Include a timezone offset only if the user explicitly mentions one."},
"end": {"type": "string", "description": "Optional end date or datetime"}, "end": {"type": "string", "description": "Optional end date or datetime in the user's local time"},
"duration": {"type": "integer", "description": "Optional duration in minutes (default 60, ignored if end is set or all_day is true)"}, "duration": {"type": "integer", "description": "Optional duration in minutes (default 60, ignored if end is set or all_day is true)"},
"description": {"type": "string", "description": "Optional event description"}, "description": {"type": "string", "description": "Optional event description"},
"location": {"type": "string", "description": "Optional event location"}, "location": {"type": "string", "description": "Optional event location"},
@@ -40,23 +62,21 @@ async def create_event_tool(*, user_id, arguments, **_ctx):
start_str = arguments["start"] start_str = arguments["start"]
end_str = arguments.get("end") end_str = arguments.get("end")
all_day = arguments.get("all_day", False) all_day = arguments.get("all_day", False)
if "T" not in start_str and " " not in start_str:
all_day = True
start_str = f"{start_str}T00:00:00"
try: try:
start_dt = datetime.fromisoformat(start_str) # Naive dates/datetimes are interpreted in the user's local
if start_dt.tzinfo is None: # timezone, not UTC. Storing UTC for naive inputs caused all-day
start_dt = start_dt.replace(tzinfo=timezone.utc) # events to land on the previous day for negative-offset users.
start_dt, start_was_date_only = await _parse_datetime_in_user_tz(
user_id, start_str
)
except (ValueError, TypeError): except (ValueError, TypeError):
return {"success": False, "error": f"Invalid start datetime: {start_str!r}"} return {"success": False, "error": f"Invalid start datetime: {start_str!r}"}
if start_was_date_only:
all_day = True
end_dt = None end_dt = None
if end_str: if end_str:
if "T" not in end_str and " " not in end_str:
end_str = f"{end_str}T00:00:00"
try: try:
end_dt = datetime.fromisoformat(end_str) end_dt, _ = await _parse_datetime_in_user_tz(user_id, end_str)
if end_dt.tzinfo is None:
end_dt = end_dt.replace(tzinfo=timezone.utc)
except (ValueError, TypeError): except (ValueError, TypeError):
return {"success": False, "error": f"Invalid end datetime: {end_str!r}"} return {"success": False, "error": f"Invalid end datetime: {end_str!r}"}
project_id = None project_id = None
@@ -86,25 +106,30 @@ async def create_event_tool(*, user_id, arguments, **_ctx):
@tool( @tool(
name="list_events", name="list_events",
description="List calendar events in a date range. Use this when the user asks what events or meetings they have. Always use full-day UTC ranges: date_from at T00:00:00Z and date_to at T23:59:59Z for the days of interest so events stored in UTC are not missed.", description="List calendar events in a date range. Use this when the user asks what events or meetings they have. Pass plain local dates (YYYY-MM-DD) — the server interprets them in the user's timezone and expands to a full local day.",
parameters={ parameters={
"date_from": {"type": "string", "description": "Start of range in ISO 8601 UTC format (e.g. 2025-01-15T00:00:00Z). Use T00:00:00Z for the start of the day."}, "date_from": {"type": "string", "description": "Start of range as a local date (YYYY-MM-DD) or local datetime. Interpreted in the user's timezone."},
"date_to": {"type": "string", "description": "End of range in ISO 8601 UTC format (e.g. 2025-01-22T23:59:59Z). Use T23:59:59Z for the end of the day."}, "date_to": {"type": "string", "description": "End of range as a local date (YYYY-MM-DD) or local datetime. A bare date is expanded to 23:59:59 local."},
}, },
required=["date_from", "date_to"], required=["date_from", "date_to"],
read_only=True, read_only=True,
briefing=True, briefing=True,
) )
async def list_events_tool(*, user_id, arguments, **_ctx): async def list_events_tool(*, user_id, arguments, **_ctx):
# Bare local dates are expanded to a full local day in the user's TZ:
# date_from → 00:00 local, date_to → 23:59:59 local, both converted to
# UTC before the DB query. Previously the tool description told the
# model to pass UTC ranges, which missed events for non-UTC users.
try: try:
date_from = datetime.fromisoformat(arguments["date_from"].replace("Z", "+00:00")) date_from, _ = await _parse_datetime_in_user_tz(
date_to = datetime.fromisoformat(arguments["date_to"].replace("Z", "+00:00")) user_id, arguments["date_from"]
)
date_to_str = arguments["date_to"]
if "T" not in date_to_str and " " not in date_to_str:
date_to_str = f"{date_to_str}T23:59:59"
date_to, _ = await _parse_datetime_in_user_tz(user_id, date_to_str)
except (ValueError, TypeError, KeyError) as exc: except (ValueError, TypeError, KeyError) as exc:
return {"success": False, "error": f"Invalid date range: {exc}"} return {"success": False, "error": f"Invalid date range: {exc}"}
if date_from.tzinfo is None:
date_from = date_from.replace(tzinfo=timezone.utc)
if date_to.tzinfo is None:
date_to = date_to.replace(tzinfo=timezone.utc)
events = await events_list_events(user_id=user_id, date_from=date_from, date_to=date_to) events = await events_list_events(user_id=user_id, date_from=date_from, date_to=date_to)
return { return {
"success": True, "success": True,
@@ -177,9 +202,9 @@ async def update_event_tool(*, user_id, arguments, **_ctx):
val = arguments.get(key) val = arguments.get(key)
if val: if val:
try: try:
dt = datetime.fromisoformat(val.replace("Z", "+00:00")) # Naive datetimes are user-local, not UTC — see
if dt.tzinfo is None: # ``_parse_datetime_in_user_tz`` docstring.
dt = dt.replace(tzinfo=timezone.utc) dt, _ = await _parse_datetime_in_user_tz(user_id, val)
fields[dt_field] = dt fields[dt_field] = dt
except (ValueError, TypeError): except (ValueError, TypeError):
return {"success": False, "error": f"Invalid datetime for {key}: {val!r}"} return {"success": False, "error": f"Invalid datetime for {key}: {val!r}"}
+47
View File
@@ -0,0 +1,47 @@
"""User-timezone helpers.
All datetimes in the DB are stored as UTC. The helpers here bridge between
that UTC storage and the user's configured local timezone (IANA string in
the ``user_timezone`` setting). Use these anywhere the model or UI talks
in terms of "today", "tomorrow", or a bare calendar date — never
``date.today()`` or ``datetime.now(timezone.utc)`` inside a per-user flow.
"""
from __future__ import annotations
from datetime import date, datetime, timedelta
from zoneinfo import ZoneInfo, ZoneInfoNotFoundError
from fabledassistant.services.settings import get_setting
# Briefing day boundary — kept in sync with the compilation slot in
# ``briefing_scheduler.SLOTS``. The briefing day flips at this local hour
# (not midnight) so the 00:0004:00 local window still shows yesterday's
# briefing until the 4am compilation generates the new one.
BRIEFING_DAY_START_HOUR = 4
async def get_user_tz(user_id: int) -> ZoneInfo:
"""Return the user's IANA ``ZoneInfo``, falling back to UTC."""
tz_str = await get_setting(user_id, "user_timezone") or "UTC"
try:
return ZoneInfo(tz_str)
except (ZoneInfoNotFoundError, KeyError):
return ZoneInfo("UTC")
async def user_today(user_id: int) -> date:
"""Return today's calendar date in the user's local timezone."""
tz = await get_user_tz(user_id)
return datetime.now(tz).date()
async def user_briefing_date(user_id: int) -> date:
"""Return the current "briefing day" in the user's local timezone.
The briefing day flips at ``BRIEFING_DAY_START_HOUR`` (4am local),
aligned with the compilation slot that generates the day's briefing.
Between 00:00 and 04:00 local this still returns *yesterday*, so the
UI keeps showing the in-progress briefing until the new one is built.
"""
tz = await get_user_tz(user_id)
return (datetime.now(tz) - timedelta(hours=BRIEFING_DAY_START_HOUR)).date()
+80
View File
@@ -134,3 +134,83 @@ def test_history_builder_no_tool_calls_unchanged():
assert len(history) == 2 assert len(history) == 2
assert history[0] == {"role": "user", "content": "Hello"} assert history[0] == {"role": "user", "content": "Hello"}
assert history[1] == {"role": "assistant", "content": "Hi there!"} assert history[1] == {"role": "assistant", "content": "Hi there!"}
# ---------------------------------------------------------------------------
# prepare_article_context tests
# ---------------------------------------------------------------------------
@pytest.mark.asyncio
async def test_prepare_article_context_small_passthrough():
"""Articles under CHAR_BUDGET pass through unchanged with zero LLM calls."""
from fabledassistant.services import article_context
body = "A short article.\n\nWith two paragraphs."
with patch(
"fabledassistant.services.article_context.generate_completion",
new_callable=AsyncMock,
) as mock_gen:
out = await article_context.prepare_article_context(
"Title", "https://example.com", body, "test-model",
)
assert out == body
mock_gen.assert_not_called()
@pytest.mark.asyncio
async def test_prepare_article_context_large_runs_map_reduce():
"""Articles over CHAR_BUDGET are chunked and map-reduced via the background model."""
from fabledassistant.services import article_context
# CHAR_BUDGET is 48_000 — build a body well over that with paragraph breaks
# so the chunker has natural splits to work with.
paragraph = ("Lorem ipsum dolor sit amet, consectetur adipiscing elit. " * 40).strip()
body = "\n\n".join([paragraph] * 30) # ~70k+ chars across 30 paragraphs
assert len(body) > article_context.CHAR_BUDGET
with patch(
"fabledassistant.services.article_context.generate_completion",
new_callable=AsyncMock,
return_value="Summary of this section with specific claims preserved.",
) as mock_gen:
out = await article_context.prepare_article_context(
"Long Article", "https://example.com/long", body, "test-model",
)
# At least one LLM call fired (the map step ran)
assert mock_gen.await_count >= 1
# Output carries the oversized-article header and section markers
assert "longer than the chat window" in out
assert "## Section 1" in out
# Map output is much smaller than the raw body
assert len(out) < len(body)
def test_chunk_by_paragraph_respects_boundaries():
"""Chunker splits on paragraph breaks, not mid-sentence."""
from fabledassistant.services.article_context import _chunk_by_paragraph, CHUNK_CHARS
paragraphs = [f"Paragraph {i}. " + ("x" * 1000) for i in range(20)]
body = "\n\n".join(paragraphs)
chunks = _chunk_by_paragraph(body)
# Each chunk stays under the budget
for chunk in chunks:
assert len(chunk) <= CHUNK_CHARS
# Total content is preserved (modulo overlap duplication, so ≥ original)
assert sum(len(c) for c in chunks) >= len(body) * 0.9
def test_chunk_by_paragraph_handles_oversized_paragraph():
"""A single paragraph larger than CHUNK_CHARS gets split mid-paragraph."""
from fabledassistant.services.article_context import _chunk_by_paragraph, CHUNK_CHARS
body = "x" * (CHUNK_CHARS * 3) # one huge paragraph, no breaks
chunks = _chunk_by_paragraph(body)
assert len(chunks) >= 3
for chunk in chunks:
assert len(chunk) <= CHUNK_CHARS
+134
View File
@@ -0,0 +1,134 @@
"""Regression tests: calendar tool respects the user's configured timezone."""
from datetime import timezone
from unittest.mock import AsyncMock, patch
import pytest
@pytest.mark.asyncio
async def test_create_event_all_day_bare_date_interprets_local_midnight():
"""Bare date like '2026-09-30' for a NY user = 2026-09-30 00:00 NY,
stored as 2026-09-30 04:00 UTC — NOT 2026-09-30 00:00 UTC (which would
be 2026-09-29 19:00 NY and land on the wrong day)."""
from fabledassistant.services.tools.calendar import create_event_tool
captured = {}
async def fake_create_event(**kwargs):
captured.update(kwargs)
event = AsyncMock()
event.to_dict.return_value = {"id": 1, **{k: v for k, v in kwargs.items() if not callable(v)}}
return event
with patch(
"fabledassistant.services.tools.calendar.get_user_tz",
AsyncMock(return_value=__import__("zoneinfo").ZoneInfo("America/New_York")),
), patch(
"fabledassistant.services.tools.calendar.events_create_event",
side_effect=fake_create_event,
):
result = await create_event_tool(
user_id=1,
arguments={"title": "Birthday", "start": "2026-09-30"},
)
assert result["success"] is True
start_dt = captured["start_dt"]
assert start_dt.tzinfo is not None
# Converted to UTC: 00:00 NY (EDT, UTC-4) == 04:00 UTC on the same day
utc = start_dt.astimezone(timezone.utc)
assert (utc.year, utc.month, utc.day) == (2026, 9, 30)
assert utc.hour == 4 # EDT offset; would be 5 in EST
assert captured["all_day"] is True
@pytest.mark.asyncio
async def test_create_event_naive_datetime_is_user_local():
"""'2026-03-15T14:30' for a NY user means 2:30pm NY, not 2:30pm UTC."""
from fabledassistant.services.tools.calendar import create_event_tool
captured = {}
async def fake_create_event(**kwargs):
captured.update(kwargs)
event = AsyncMock()
event.to_dict.return_value = {"id": 1}
return event
with patch(
"fabledassistant.services.tools.calendar.get_user_tz",
AsyncMock(return_value=__import__("zoneinfo").ZoneInfo("America/New_York")),
), patch(
"fabledassistant.services.tools.calendar.events_create_event",
side_effect=fake_create_event,
):
await create_event_tool(
user_id=1,
arguments={"title": "Meeting", "start": "2026-03-15T14:30"},
)
utc = captured["start_dt"].astimezone(timezone.utc)
# 14:30 NY on 2026-03-15 (after DST start) = 18:30 UTC
assert (utc.year, utc.month, utc.day, utc.hour, utc.minute) == (2026, 3, 15, 18, 30)
@pytest.mark.asyncio
async def test_create_event_explicit_offset_is_respected():
"""Model-supplied timezone offsets must not be overridden by user TZ."""
from fabledassistant.services.tools.calendar import create_event_tool
captured = {}
async def fake_create_event(**kwargs):
captured.update(kwargs)
event = AsyncMock()
event.to_dict.return_value = {"id": 1}
return event
with patch(
"fabledassistant.services.tools.calendar.get_user_tz",
AsyncMock(return_value=__import__("zoneinfo").ZoneInfo("America/New_York")),
), patch(
"fabledassistant.services.tools.calendar.events_create_event",
side_effect=fake_create_event,
):
await create_event_tool(
user_id=1,
arguments={"title": "Meeting", "start": "2026-03-15T14:30+00:00"},
)
utc = captured["start_dt"].astimezone(timezone.utc)
assert (utc.hour, utc.minute) == (14, 30)
@pytest.mark.asyncio
async def test_list_events_bare_date_range_covers_local_day():
"""date_from/date_to as bare dates should cover the full LOCAL day."""
from fabledassistant.services.tools.calendar import list_events_tool
captured = {}
async def fake_list_events(*, user_id, date_from, date_to):
captured["date_from"] = date_from
captured["date_to"] = date_to
return []
with patch(
"fabledassistant.services.tools.calendar.get_user_tz",
AsyncMock(return_value=__import__("zoneinfo").ZoneInfo("America/New_York")),
), patch(
"fabledassistant.services.tools.calendar.events_list_events",
side_effect=fake_list_events,
):
await list_events_tool(
user_id=1,
arguments={"date_from": "2026-09-30", "date_to": "2026-09-30"},
)
df = captured["date_from"].astimezone(timezone.utc)
dt = captured["date_to"].astimezone(timezone.utc)
# 2026-09-30 00:00 NY (EDT) = 04:00 UTC same day
assert (df.year, df.month, df.day, df.hour) == (2026, 9, 30, 4)
# 2026-09-30 23:59:59 NY (EDT) = 03:59:59 UTC next day
assert (dt.year, dt.month, dt.day, dt.hour) == (2026, 10, 1, 3)
+63
View File
@@ -0,0 +1,63 @@
"""Tests for user-timezone helpers (services/tz.py)."""
from datetime import datetime
from unittest.mock import AsyncMock, patch
from zoneinfo import ZoneInfo
import pytest
@pytest.mark.asyncio
async def test_get_user_tz_returns_configured_zone():
from fabledassistant.services import tz as tz_mod
with patch.object(tz_mod, "get_setting", AsyncMock(return_value="America/New_York")):
zone = await tz_mod.get_user_tz(1)
assert zone == ZoneInfo("America/New_York")
@pytest.mark.asyncio
async def test_get_user_tz_falls_back_to_utc_on_bad_input():
from fabledassistant.services import tz as tz_mod
with patch.object(tz_mod, "get_setting", AsyncMock(return_value="Not/AZone")):
zone = await tz_mod.get_user_tz(1)
assert zone == ZoneInfo("UTC")
@pytest.mark.asyncio
async def test_user_briefing_date_before_4am_returns_yesterday():
"""00:0003:59 local still shows yesterday's briefing day."""
from fabledassistant.services import tz as tz_mod
ny = ZoneInfo("America/New_York")
# 02:00 NY on 2026-04-13 → briefing day = 2026-04-12
fake_now = datetime(2026, 4, 13, 2, 0, tzinfo=ny)
class _FakeDatetime(datetime):
@classmethod
def now(cls, tz=None):
return fake_now
with patch.object(tz_mod, "get_user_tz", AsyncMock(return_value=ny)), \
patch.object(tz_mod, "datetime", _FakeDatetime):
day = await tz_mod.user_briefing_date(1)
assert day.isoformat() == "2026-04-12"
@pytest.mark.asyncio
async def test_user_briefing_date_after_4am_returns_today():
from fabledassistant.services import tz as tz_mod
ny = ZoneInfo("America/New_York")
fake_now = datetime(2026, 4, 13, 4, 30, tzinfo=ny)
class _FakeDatetime(datetime):
@classmethod
def now(cls, tz=None):
return fake_now
with patch.object(tz_mod, "get_user_tz", AsyncMock(return_value=ny)), \
patch.object(tz_mod, "datetime", _FakeDatetime):
day = await tz_mod.user_briefing_date(1)
assert day.isoformat() == "2026-04-13"