Two related reliability fixes.
1. routes/chat.py — guard run_generation against uncaught exceptions.
run_generation is launched with asyncio.create_task(); any exception
raised inside the coroutine is silently swallowed by the event loop,
the buffer stays in GenerationState.RUNNING forever, and every
subsequent POST /api/chat/conversations/<id>/messages returns 409
'Generation already in progress' — locking the user out of the chat
with no log trail.
Observed in dev 2026-05-22: assistant message 768 created at 20:36:59
with status=generating, stayed in that state for an hour+, and four
follow-up message attempts returned 409 instantly. The generation
task hung before any internal log line could fire, so the only
diagnostic was the 409 responses themselves.
Wrap run_generation in _run_generation_guarded() that catches
exceptions, logs with full traceback, transitions the buffer to
ERRORED, emits a final 'done' SSE event so any active stream
client closes cleanly, and marks the assistant message status=error
in the DB. After this, a stuck conversation recovers on its own
the next time the user sends a message — no manual DB poke needed.
2. services/curator_scheduler.py — pass last_curator_run_at as 'since'
to the curator so each sweep only sees messages added after the
previous successful pass.
Previously the scheduler called run_curator_for_conversation(conv_id)
with no 'since' argument, so the curator defaulted to its 24h
lookback window. Within an active journal session that meant every
15-min sweep re-extracted beats from messages already captured
on prior sweeps — producing duplicate moments.
_candidate_conversations() now returns (conv_id, last_curator_run_at)
tuples; _sweep() threads the timestamp through. First-run case
(last_curator_run_at IS NULL) falls back to the curator's default
24h window, which is what we want — process recent backlog on
first contact, then only deltas after.
Manual trigger path (POST /api/journal/curator/run/<conv_id>) is
intentionally NOT changed; it still passes since=None so the
24h re-sweep behaviour is preserved for ad-hoc 'reprocess today'
clicks from the UI.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Removes the entire RSS feature surface — feeds, items, embeddings, reactions,
discussion-note flow, briefing news context, settings, env-vars, and DB
tables. Keeps the URL-generic article-reader (the read_article LLM tool)
under a clean module so the LLM can still fetch arbitrary article content
from URLs the user provides.
Backend:
- New services/article_fetcher.py — single source of trafilatura URL→text
- New services/tools/article.py — read_article tool (was nested under tools/rss)
- Delete services/rss.py, rss_classifier.py, rss_filtering.py, article_context.py
- Delete services/tools/rss.py
- Delete models/rss_feed.py (RssFeed, RssItem), models/rss_item_embedding.py
- services/embeddings.py: drop upsert/semantic_search/backfill RSS helpers
- services/llm.py: remove _build_briefing_article_context, briefing-conv branch,
ARTICLE_DISCUSS_SEED skip-RAG branch; drop get_rss_items / add_rss_feed from
the actions list
- services/generation_task.py: drop _maybe_save_article_discussion_note + caller
- routes/chat.py: drop /api/chat/from-article/<id> endpoint
- routes/journal.py: re-import via web.py refactor (article_fetcher path)
- services/tools/__init__.py: register `article`, drop `rss`
- services/tools/_registry.py: drop the requires=='rss' check
- app.py: drop backfill_rss_item_embeddings + backfill_rss_article_content tasks
- config.py: prose-only edit (no env var change — RSS env vars were never first-class)
Frontend:
- stores/settings.ts: drop rssEnabled
- SettingsView.vue: drop the RSS-classification mention
- api/client.ts: drop openArticleInChat (the from-article endpoint is gone)
Tests:
- Delete tests/test_rss_service.py, test_news_api.py, test_article_reading.py
Migration:
- 0042_drop_rss: DROP TABLE rss_item_embeddings, rss_item_reactions, rss_items,
rss_feeds; DELETE settings rows for rss_enabled / briefing_*_topics
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Discuss flow was hallucinating unrelated content when article
extraction returned empty or RAG pulled in orphan notes that looked
more relevant than the generic seed prompt.
- seed_article_discussion raises EmptyArticleError on empty body;
briefing and /news routes return 422 instead of staging an empty
synthetic tool result.
- build_context skips RAG auto-injection when user_message matches
ARTICLE_DISCUSS_SEED so the article IS the context on turn one;
follow-up turns keep RAG on.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Both the /news discuss button and the briefing discuss button now call a
shared seed_article_discussion() helper that stages the synthetic
read_article tool exchange and the conversational seed prompt — behavior
stays byte-identical across entry points. /news also auto-starts
generation so the chat screen lands on an in-flight stream.
First assistant reply in a seeded article conversation is persisted as a
Note (tags: article-summary + article topics) and backlinked via
rss_items.discussion_note_id, so the knowledge base stops being amnesiac
about articles the user has engaged with.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace the hardcoded "2h" keep_alive everywhere with a helper that
returns OLLAMA_KEEP_ALIVE_MAIN (default 30m) for the interactive model
and OLLAMA_KEEP_ALIVE_BACKGROUND (default 10m) for the background
model. Lets the main model release VRAM during long idle periods
while keeping it warm enough for bursty chat use, and stops the
sporadic background model from camping VRAM it rarely needs.
Seven call sites updated to route through llm.keep_alive_for(model):
the streaming helpers, generate_completion, the two startup warmers,
the settings KV-cache primer, and the chat warmer endpoint.
Override via env vars: OLLAMA_KEEP_ALIVE_MAIN, OLLAMA_KEEP_ALIVE_BACKGROUND.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The history builder was silently dropping tool_calls from prior turns,
causing the LLM to lose article/search context on every follow-up.
Now reconstructs the assistant tool_call dict + per-call tool result
entries. Messages without tool_calls are unaffected.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- cleanup_old_conversations now excludes briefing conversations (was
silently deleting briefing history after the retention window)
- list_conversations response now includes rag_project_id, matching the
shape returned by the single-conversation GET endpoint
- create_conversation_from_article: removed duplicate async_session import
(_session2 was a copy of the same import); consolidated into one
- MAX_TOOL_ROUNDS fixed from 5→6 to match the actual range(6) loop;
loop updated to range(MAX_TOOL_ROUNDS) so the constant is accurate
- Chat retention cleanup moved from per-request (every GET /conversations)
to a daily scheduled job in event_scheduler.py; route no longer runs
a DB write on every read
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Embed RSS items at fetch time (nomic-embed-text); backfill at startup
- Semantic news search injected into chat system prompt ("Recent News You've Seen")
when items match query above 0.55 cosine threshold (independent of note RAG)
- "Discuss in chat" button on news cards — creates a seeded conversation with
the article title + full content, navigates directly to the new chat
- Briefing compilation now passes 500-char article excerpts (not just headlines)
to the LLM and uses 8192 num_ctx to accommodate the larger prompt
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Add parse_pagination() to routes/utils.py; replace 6 duplicate limit/offset extractions in notes, tasks, chat, projects, milestones routes
- Extract _enrich_shares() in sharing.py; eliminates identical 12-line loop in list_project_shares and list_note_shares
- Extract _deduplicate_by_permission() in sharing.py; eliminates identical deduplication blocks in list_shared_with_me for projects and notes
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Browser timezone is now synced to user_settings["user_timezone"] on
every login/page load (App.vue). The briefing scheduler and LLM context
both read from this single source, falling back to the legacy
briefing_config.timezone for existing users during migration.
- App.vue: PUT /api/settings with browser IANA timezone on startAppServices
- routes/chat.py: fall back to stored user_timezone when not sent in request
- briefing_scheduler: read user_timezone setting; briefing_config.timezone
kept as fallback only
- routes/briefing.py: pass tz_override from user_timezone to live-patched scheduler
- Remove timezone field from BriefingConfig interface and all briefing UI
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Frontend sends user_timezone (IANA, from Intl.DateTimeFormat) with
every message POST; threaded through route → generation_task → build_context
- System prompt now tells the LLM the user's timezone so it creates
events with the correct UTC offset (e.g. 15:00+01:00 not 15:00Z)
- Calendar tool guidance updated to require UTC offset in all event
datetimes
- EventSlideOver: dateFromIso/timeFromIso now use JS Date to convert
stored UTC times to local time for display; toIso includes local
timezone offset when saving so the correct UTC time is stored
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Backend:
- Enrich GET /api/chat/models to also hit /api/ps and return loaded:bool
and modified_at alongside name/size, using parallel gather
Frontend (Settings → General):
- Model list: each row shows name (monospace), size (GB/MB), 'in VRAM' badge
if currently loaded, 'default' badge if it's the configured default
- Delete button per row; disabled while deletion in progress
- Pull form: text input (Enter submits) + Pull button
- Suggestion chips for qwen3:7b/14b/4b, llama3.1:8b, nomic-embed-text;
disabled if already installed
- Progress display during pull: status text + determinate bar when
Ollama reports total/completed, indeterminate animation otherwise
- Refresh button reloads the list; list auto-refreshes after pull/delete
- Link to ollama.com/library for model discovery
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
conv.model was stored at conversation creation time and short-circuited
the get_setting() call, so changing default_model in Settings had no
effect on existing conversations. Now the user's default_model setting
is always the source of truth for generation and summarization.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- stream_chat and stream_chat_with_tools: remove read=300s per-chunk
timeout, replace with read=None. In httpx streaming mode, the read
timeout applies per-chunk — if Ollama pauses >300s while processing
a large input context before the first token, it raises ReadTimeout,
killing generation and leaving the assistant message as an empty stub.
With read=None the stream is unbounded; connect=30s still guards the
initial connection.
- chat_status_route: increase Ollama status check timeout 5s → 10s.
When Ollama is busy processing a large prompt it can be slow to
respond to /api/tags, causing the status indicator to briefly flip to
"offline" even though generation is running normally.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
When a user selected the 'Default' option in Settings, the dropdown
sent an empty string "" to the backend. The route saved it as a DB row,
which caused get_setting() to return "" instead of falling back to
Config defaults. The chat status endpoint then tried to match "" against
installed model names — always failing — resulting in model: "not_found"
and a permanently failing readiness indicator.
services/settings.py:
- Add delete_setting() helper: removes a setting row so get_setting()
correctly falls back to its hardcoded default argument
routes/settings.py:
- Import delete_setting
- When default_model or intent_model are saved as empty string, delete
the DB row instead of storing "" — cleanly restores Config fallback
routes/chat.py:
- chat_status_route: add explicit `or Config.OLLAMA_MODEL` guard for
any existing "" rows written before this fix (migration safety net)
- send_message and summarize routes: same guard on model resolution
so empty settings never cause silent generation failures
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Backend security & correctness:
- Add rate_limit.py: sliding-window rate limiter (asyncio) applied to login,
register, forgot/reset password endpoints (10/60s or 5/300s per IP)
- app.py: add security headers in after_request (X-Frame-Options, CSP,
X-Content-Type-Options, Referrer-Policy) using setdefault to preserve SSE headers
- auth.py: refactor duplicate login_required/admin_required into shared _check_auth()
- config.py: add TRUST_PROXY_HEADERS for proxy-aware client IP resolution
- routes/auth.py: rate limiting, _client_ip() helper, cleaned-up reset_password route
- routes/chat.py, notes.py, tasks.py: int() DoS fix on last_event_id; limit capped
at 500; date.fromisoformat() wrapped in try/except → 400 on invalid dates
- services/auth.py: fix Setting.user_id update bug (filter on NULL not user.id);
reset_password_with_token returns int|None (user_id) instead of bool
- services/backup.py: add _security_notice to full backup JSON export
- services/assist.py: system prompt explicitly preserves markdown list structure
and nested indented sub-items
Infrastructure:
- docker-compose.yml: add healthcheck on app service (/api/health, 10s interval)
- .dockerignore: prevent secrets/node_modules/__pycache__/.env.* leaking into build
Frontend bug fixes:
- TaskCard.vue, TaskViewerView.vue: fix isOverdue() timezone bug (ISO string compare)
- useAssist.ts: accept() now resets state to idle when document changed since proposal
- stores/chat.ts: fix memory leak in _pollUntilLoaded() (try/catch around fetchStatus)
- TiptapEditor.vue: selection offset uses closest-match strategy (not first-match)
- utils/markdown.ts: explicit DOMPurify config with FORCE_BODY; remove as const
(DOMPurify expects mutable string[])
New features:
- Auto-save (5-minute interval) in NoteEditorView and TaskEditorView — only when
editing an existing dirty record; silent on error, shows "Auto-saved" toast
- sectionParser.ts: top-level bullet/numbered list items are now individual sections
in the AI Assist panel (previously treated as one undifferentiated block)
- editor-shared.css: extracted ~500 lines of CSS duplicated between both editors;
includes .inline-assist-btn at global scope (required for teleported elements)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Context sidebar + note title:
- ChatView: replace ephemeral context pills with a persistent right-panel sidebar;
auto-found notes accumulate across turns; attached note shows with pin icon;
× button excludes a note from future auto-search; hidden on mobile
- routes/chat.py: batch-fetch note titles via get_notes_by_ids() and inject
context_note_title into each message dict at conversation load time
- notes.py: add get_notes_by_ids() batch fetch helper
- types/chat.ts: add context_note_title field to Message interface
- stores/chat.ts: sendMessage accepts optional 5th arg contextNoteTitle,
included in optimistic user message
- ChatMessage.vue: context badge shows note title instead of 'Note #N'
Expanded LLM tool suite (all with intent router rules + ToolCallCard display):
- delete_note / delete_task: permanent delete with user confirmation (write tool),
type-safe (refuse to delete wrong type), clears note context cache on success
- get_note: fetch full note body by query (search_notes returns only 200-char preview)
- list_notes: browse notes by recency/keyword/tags with limit; notes only
- update_note: add tags + tag_mode (replace/add/remove) parameters
- search_notes: add optional type filter ("note" | "task")
- search_todos (CalDAV): keyword-filter todos, companion to list_todos
- caldav.py: add search_todos() built on top of list_todos()
- generation_task.py: register new tools in _WRITE_TOOLS, _TOOL_LABELS, _TOOL_ACTIONS
- llm.py: update available actions list and guidance in system prompt
- intent.py: routing rules for all new tools
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
stream_chat_with_tools now accepts a think parameter. run_generation
forwards it to Ollama. The message POST route reads think from the
request body. ChatView passes think=true so qwen3 uses chain-of-thought
reasoning for full conversations; the dashboard widget and ChatPanel
omit it, staying fast. Dashboard button updated to "Think it through
in Chat →" to signal the deeper capability.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Before executing any write tool (create/update/delete), the backend now
pauses with an asyncio.Future and emits a tool_pending SSE event. The
frontend displays a ToolConfirmCard with Accept and Decline buttons.
Clicking Accept resolves the Future and proceeds; Decline records a
declined tool_call chip and falls through to regular streaming. Typing
single-word yes/no responses (e.g. "yes", "cancel") also works as
confirmation. 120s timeout auto-declines if no response.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- build_context() moved from route handler into run_generation() background task.
The 202 response now returns immediately; client connects to SSE before
note search / URL fetch begins, so 'Building context...' status is visible.
- _generate_title() runs in a fire-and-forget asyncio.create_task() after the
'done' SSE event fires. Users see their response complete 2–5s sooner on new
conversations; title appears later in the sidebar without blocking the stream.
- generate_completion() now sets think:False and accepts a max_tokens limit.
Intent classifier passes max_tokens=200 (JSON only), title generator passes
max_tokens=30 (short title), eliminating qwen3 thinking-mode overhead on these
auxiliary calls.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Streaming status transparency:
- generation_task.py emits 'status' SSE events at each pipeline stage:
"Analyzing your request..." before intent classification, tool label
before each tool execution, "Generating/Composing response..." before
each LLM streaming round
- chat.ts adds streamingStatus ref; cleared on first chunk or done/error;
includes fast 5s poll loop after warmModel() until model shows as loaded
- ChatView.vue shows pulsing dot + italic status label above content area;
falls back to blinking cursor once content arrives
- HomeView.vue shows status label in dashboard panel instead of '...'
Model load state indicator:
- /api/chat/status now queries /api/tags and /api/ps in parallel to
distinguish installed-but-cold vs loaded-in-VRAM model states
- New model status values: 'not_found' | 'cold' | 'loaded' (was 'ready')
- chatReady true for both 'cold' and 'loaded' (cold models still work)
- AppHeader shows 5 states: gray pulse (checking), red (Ollama down),
orange (not installed), yellow pulse (cold), green (loaded)
- Inline short label ("Cold", "Ready", "Offline", etc.) visible without
hovering; detailed tooltip on hover
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Green/red emoji circles replace Unicode symbols for at-a-glance model
readiness. Increase connect timeouts (10→30s) for cold model loading,
warm timeout (120→300s) for large models, and pull timeout (600→1800s)
to match route-level limit. Show error toast on SSE reconnection failure
instead of silently recovering.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add GET /api/chat/ps and POST /api/chat/warm endpoints for hot model
visibility and pre-loading
- Extend PATCH /api/chat/conversations/:id to accept model in addition
to title
- Add ModelSelector component with hot/cold indicators from Ollama /api/ps
- Add DashboardChatInput component (model selector + note picker + textarea)
replacing the simple "New Chat" button on the dashboard
- Add model selector dropdown to ChatView header, persisted per-conversation
- Warm default model on dashboard mount via fire-and-forget background task
- Configure Ollama with OLLAMA_MAX_LOADED_MODELS=2 and OLLAMA_KEEP_ALIVE=30m
- Always-visible edit buttons on NoteCard/TaskCard (remove hover-only behavior)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Rewrite get_all_tags() with SQL unnest instead of loading all notes
- Consolidate convert_note_to_task/convert_task_to_note to single-session ops
- Add search_notes_for_context() with single OR-keyword query for build_context()
- Drop selectinload from list_conversations(), use correlated subquery for message_count
- Add set_settings_batch() for single-transaction multi-setting upserts
- Extract get_installed_models() shared helper into services/llm.py
- Delete services/tasks.py pass-through wrapper; routes/tasks.py imports from services.notes
- Add B-tree indexes on notes.title and conversations.updated_at (migration 0007)
- Add logging to services/notes.py, services/chat.py, services/settings.py
- Safe Conversation.to_dict() when messages relationship is not loaded
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Improve chat context: build_context() now returns metadata about auto-found
notes, emitted as an SSE event so the frontend can display context pills
showing which notes influenced the response. Users can promote notes for
deeper context (+) or exclude irrelevant ones (x). A note picker lets users
manually attach notes. Multi-word search uses per-term AND matching, and
auto-search iterates keywords individually for broader OR-style coverage.
Standardize styling: introduce CSS design tokens (--radius-sm/md/lg/pill,
--color-success/warning/overlay, --focus-ring) and migrate all components
to use them. Fix header alignment to full-width, add active nav link state,
replace hardcoded colors with CSS variables, and normalize button padding.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace the asyncio.create_task() fire-and-forget pattern for model
downloads with SSE streaming that relays Ollama's pull progress in
real time. The frontend now shows live download percentage instead of
a static "Pulling..." label with blind polling.
- routes/chat.py: SSE streaming endpoint with 1800s timeout
- stores/settings.ts: pullModel uses apiStreamPost with onProgress callback
- SettingsView.vue: live "Downloading 42%" display, removed polling/setTimeout
- summary.md: updated to reflect SSE streaming for model pulls
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Settings infrastructure: key-value settings table, GET/PUT API, Pinia store
- Configurable assistant name (default "Fable") in settings and LLM system prompt
- Model catalog with 18 models in 3 categories (General Purpose, Coding,
Uncensored / Creative Writing) with download/select/remove functionality
- Move Ollama status indicator from chat views to global nav bar
- Chat bubble layout: user messages right-aligned, assistant left-aligned
- Floating dark input bar with auto-focus and circular send button
- Fix HTML entity rendering (' apostrophe issue in marked/DOMPurify pipeline)
- Fix new chat button navigation (fetchConversation before router.push)
- Recent chats section on home page with "New Chat" button
- Update summary.md with Phase 4.5 changes
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Adds visibility into whether Ollama is reachable and the configured model
is ready before allowing chat. Prevents silent failures when the model is
still downloading or Ollama is unavailable.
- Backend: GET /api/chat/status endpoint checks Ollama /api/tags (5s timeout)
Returns ollama availability + model readiness + default model name
- Frontend: OllamaStatus type, chat store polling (30s interval)
- ChatView: status dot + label in header (green/yellow/red), disabled send
- ChatPanel: compact status indicator in panel header, disabled send
- summary.md: updated with new endpoint, store changes, and component descriptions
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Phase 4: Full chat system with SSE streaming, note-aware context, and
conversation persistence.
Backend:
- Migration 0005: conversations + messages tables with FKs and indexes
- Conversation/Message SQLAlchemy models with relationships
- LLM service: ensure_model (auto-pull on startup), stream_chat (NDJSON),
generate_completion, fetch_url_content (HTML stripping), build_context
(keyword extraction, related note search, URL content injection)
- Chat service: conversation CRUD, save_response_as_note,
summarize_conversation_as_note
- Chat routes blueprint: 9 endpoints including SSE streaming for messages,
save/summarize as note, Ollama model listing
- Auto-pull llama3.1 model on app startup (non-blocking)
Frontend:
- apiStreamPost: SSE client using fetch + ReadableStream
- Chat Pinia store with streaming state management
- ChatView: dedicated /chat page with conversation sidebar + message thread
- ChatPanel: slide-out panel with contextNoteId from current route
- ChatMessage: markdown-rendered message bubble with "Save as Note" action
- Updated AppHeader with Chat nav link + panel toggle button
- Updated App.vue to mount ChatPanel with route-derived context
- Added /chat and /chat/:id routes
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
bvandeusen
Bryan Van Deusen