Commit Graph

254 Commits

Author SHA1 Message Date
bvandeusen 8762552234 feat(briefing): project continuity — yesterday's activity and stale project warnings 2026-04-08 22:19:38 -04:00
bvandeusen 6cbf9be052 feat(briefing): weekly review — 7-day recap with task/note/project summary and upcoming preview 2026-04-08 22:18:19 -04:00
bvandeusen 3687fbeeb9 feat(briefing): differentiated check-in slots — midday progress nudge, afternoon wrap-up 2026-04-08 22:09:18 -04:00
bvandeusen a6543c1dc5 feat(briefing): cluster news by topic for thematic synthesis instead of flat headlines 2026-04-08 22:07:24 -04:00
bvandeusen 4558dd578a feat(briefing): weather × calendar cross-reference — rain warnings for outdoor events 2026-04-08 22:03:18 -04:00
bvandeusen 8647f52fbc feat(weather): live current conditions endpoint + polling display in briefing view 2026-04-08 21:33:27 -04:00
bvandeusen 304affb837 feat(briefing): daily planning — prioritized focus list and calendar gap analysis 2026-04-08 20:59:06 -04:00
bvandeusen 2e3f90384d feat(briefing): actionable intelligence — days overdue, project context, suggest next steps 2026-04-08 20:33:16 -04:00
bvandeusen f2debd8a5b feat(knowledge): show organization for person cards, category for place cards
Exposes birthday, organization, address (person) and website, category (place)
from entity_meta in the knowledge API response; updates KnowledgeView cards
to display organization and category as visible meta chips/text.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-08 13:32:15 -04:00
bvandeusen dd16b39218 feat(knowledge): include tasks in knowledge queries and counts 2026-04-08 10:27:31 -04:00
bvandeusen 39e554d938 fix: rewrite title generator to only use user messages; bump background model to qwen2.5:3b 2026-04-07 12:53:33 -04:00
bvandeusen b3cf42863a fix: reinforce no-project-inference in system prompt; filter tool messages from title generation 2026-04-07 12:45:08 -04:00
bvandeusen d290bebad2 feat(stt): pass conversation context as Whisper initial_prompt to reduce mishearings 2026-04-07 08:48:27 -04:00
bvandeusen d3170e5545 fix(chat): fetch full article content in from-article endpoint 2026-04-07 08:13:40 -04:00
bvandeusen 814f44c3fb fix(push): fix VAPID key format and add regenerate endpoint + UI button 2026-04-07 06:59:06 -04:00
bvandeusen 1d0cf4828b fix(briefing): fetch full article content via trafilatura in discuss endpoint 2026-04-07 06:55:15 -04:00
bvandeusen edf0a9063e fix: prevent model from inferring project names on create_task/create_note
The model was hallucinating project names from task/note content (e.g.
inferring "Vehicle Maintenance" from "purchase wheel bearings"). Added
explicit guidance to both project field descriptions: only set if the
user explicitly named a project, never infer from content.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-06 19:17:43 -04:00
bvandeusen 9af8ab8f70 fix(briefing): use briefing context for follow-ups; add slot separator
- build_context: when conversation_type is 'briefing', inject a system
  prompt instruction telling the model to answer from conversation history
  and article context instead of searching the web
- Consolidate briefing conversation type detection to one DB query (was
  being checked twice — once for the system prompt addition, once for
  article context injection)
- ChatPanel: render a visual 'New Briefing Update' separator line before
  2nd+ briefing slot messages (identified by metadata.rss_item_ids)
- types/chat.ts: add metadata field to Message interface

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-06 06:15:18 -04:00
bvandeusen a171210224 feat(tools): confirmed guard for deletes, update_person/place, get/update_profile, calculate
- delete_note / delete_task: add confirmed parameter + requires_confirmation guard
  (find the note first, then ask, consistent with create_note/task pattern)
- get_note: description now mentions notes AND tasks
- update_person / update_place: new tools to update existing entity notes in-place
- get_profile / update_profile: surface and edit the user's stored profile
  (expertise, tone, response style, job title, interests)
- calculate: eval math expressions via Python math module; solves precision issues
  on multi-step arithmetic and supports sqrt/log/trig/etc.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-05 22:58:14 -04:00
bvandeusen eb92b2a976 feat(research): multi-note pipeline — outline + parallel section synthesis + index note
Replaces the single monolithic research note with topic-driven section notes
plus an index note. Two new LLM calls: _generate_outline (JSON outline, 3-8
sections) and _synthesize_section (300-600 word focused note per section,
parallelised via asyncio.gather). Public signature of run_research_pipeline
unchanged; falls back to single-note synthesis on outline failure or if all
sections fail.

Also extracts _build_sources_block helper and adds full test suite.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-05 22:53:14 -04:00
bvandeusen e4c812a603 feat(voice): improve TTS logging for root-cause diagnosis
- Route now logs every synthesis request (char count, voice, speed)
- Route logs char count + text preview when the 8000-char limit is hit
- Route logs empty audio with preview (helps spot no-chunk-produced edge case)
- Route logs success with byte count and duration
- Kokoro synthesise() logs per-call: samples produced, elapsed, chars/s
- Kokoro synthesise() logs warning when zero audio chunks returned with preview
- Kokoro synthesise() catches and logs pipeline-internal errors with preview
- Frontend: console.warn now includes char count + 80-char preview on failure and retry

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-05 22:36:43 -04:00
bvandeusen 3bdadaeca8 fix(embeddings): remove stale CONTENT_MAX_CHARS import from rss.py
CONTENT_MAX_CHARS was removed from rss.py when the article content cap
was lifted. backfill_rss_article_content still referenced it, causing an
ImportError on startup.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-05 22:04:10 -04:00
bvandeusen 8ec91ceea7 feat(settings): propagate user_timezone to briefing scheduler on save
When user_timezone is saved via PUT /api/settings, immediately call
update_user_schedule if briefing is enabled so the scheduler picks up
the new timezone without requiring a restart.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-05 21:35:00 -04:00
bvandeusen 0ec030cb8f feat(scheduler): slot gating + morning work-day gate
- _add_user_jobs now accepts config dict and skips disabled slots
- _get_briefing_enabled_users returns 3-tuple (user_id, tz, config)
- update_user_schedule passes config through to _add_user_jobs
- _run_slot_for_user skips morning slot on non-work days via profile.work_schedule
- Add tests for slot gating and work-day gate

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-05 21:34:29 -04:00
bvandeusen 68eee57c9b refactor(quick-capture): replace intent router with native tool-calling
Removes the custom classify_capture_intent + _process_note two-pass
approach. The LLM now picks the right tool directly via Ollama's native
tool_calls API (same path as the main chat pipeline). _should_think
decides whether extended reasoning is needed based on input length/
complexity. intent.py deleted — no longer needed.

Android app and response format unchanged.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-05 19:49:44 -04:00
bvandeusen 284dcd1c63 feat(briefing): add article Discuss endpoint with synthetic tool exchange
POST /api/briefing/articles/<id>/discuss injects stored article content
as a persisted read_article tool exchange before triggering generation.
The LLM sees the article as already read; follow-ups retain context via
the fixed history builder. Frontend Discuss button now calls the new
endpoint instead of inlining article text in the user message.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-05 19:36:45 -04:00
bvandeusen 7dc5af2e88 feat(chat): add tool_calls param to add_message for synthetic messages
Needed by the Discuss endpoint to persist synthetic read_article
tool exchanges before triggering generation.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-05 19:33:54 -04:00
bvandeusen eeb671872a fix(chat): replay tool_calls in history so tool context survives follow-ups
The history builder was silently dropping tool_calls from prior turns,
causing the LLM to lose article/search context on every follow-up.
Now reconstructs the assistant tool_call dict + per-call tool result
entries. Messages without tool_calls are unaffected.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-05 19:33:31 -04:00
bvandeusen 278927ec40 feat(tools): add read_article tool for fetching full article content
Adds read_article tool definition and execute_tool handler. Uses
_fetch_full_article (trafilatura) from rss.py, caps tool output at
40K chars to keep context window manageable. Always registered
(not gated on SearXNG). Tests cover success, failure, truncation,
empty URL, and history builder replay.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-05 19:33:11 -04:00
bvandeusen f7d54a15c0 feat(rss): remove article content character cap
Trafilatura extracts only article body text (typically 2K–15K chars),
so storing the full content is safe without an artificial ceiling.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-05 19:32:17 -04:00
bvandeusen db092b113e fix: rss classifier think-tag stripping, briefing calendar dict access, embed empty-string guard
- rss_classifier: strip <think>...</think> blocks (qwen3 reasoning output)
  before JSON parse; use strict=False for control chars; bump timeout 30s→120s
- briefing_pipeline: list_events returns dicts not Event objects — fix
  attribute access (.all_day/.start_dt/.title) to dict access
- embeddings: guard upsert_note_embedding and semantic_search_notes against
  empty-string input to prevent Ollama embed 400 errors

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-05 13:14:55 -04:00
bvandeusen 94d21c4512 fix(settings): audit pass — model auto-pull on startup, background_model empty-string bug, base URL validation
- Startup now pulls Config.OLLAMA_MODEL (system default chat model) — previously only
  embedding and background models were pulled; the primary chat model was skipped
- _warm_user_models expanded to also pull user-configured default_model and
  background_model overrides that are missing from Ollama, rather than logging and
  skipping them; pulls run before warm/KV-cache priming
- Add background_model to _MODEL_KEYS in settings route so clearing the dropdown
  deletes the DB row instead of saving "", which caused Ollama failures in tag
  suggestions, title generation, project summaries, and RSS classification
- Add http/https scheme validation to PUT /api/admin/base-url matching the CalDAV
  route pattern; a bad value no longer silently breaks invite/password-reset links
- Update admin voice config description: "Reload models" button exists to avoid
  a server restart, so the old "restart required" text was misleading

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-04 14:08:14 -04:00
bvandeusen d9bd16633f fix(tasks): audit pass — permission checks, tool gaps, hard reload
- PUT/PATCH/DELETE /api/tasks/:id now use get_note_for_user + can_write_note
  so shared project editors can mutate tasks; owners unaffected
- PATCH /api/tasks/:id/status gets same treatment
- All write routes call update_note/delete_note with note.user_id (owner)
  not the accessing user's uid, matching the milestone fix pattern
- create_task tool gains tags (array) and status (enum) parameters;
  handler now passes tags to create_note and respects initial status
- create_task tool response now includes milestone_id and parent_id
- update_note tool gains milestone parameter; handler resolves the
  milestone by title within the note's current (or newly set) project,
  clears milestone_id when project is cleared
- list_tasks tool gains q keyword search parameter; passed through
  to list_notes
- TaskEditorView: replace window.location.reload() with
  router.push('/tasks/:id') after save

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-04 13:48:07 -04:00
bvandeusen c5191837fb fix(projects): audit pass — 8 correctness and consistency fixes
- Project.to_dict() now includes user_id and auto_summary
- Status validation unified to (active/completed/archived) on both
  create and update project routes; update route previously had none
- Milestone routes: replace get_project (ownership-only) with
  get_project_for_user so shared viewers/editors can access milestones
- Add get_milestone_in_project() to milestones service for project-
  scoped lookup without user_id filter; all milestone routes use it
- Milestone PATCH now validates status as 'active'|'done'; fix tool
  enum which was wrongly ['active','completed','cancelled']
- Write mutation routes (POST/PATCH/DELETE milestones) now check
  can_write_project() and return 403 for read-only shared users
- update_project tool now exposes title and color fields so projects
  can be renamed or recolored via chat
- create_project tool now exposes color field
- GET /api/projects?include_summary=true embeds summaries in one
  backend pass; ProjectListView switches to this, eliminating N+1
  per-project fetches

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-04 13:14:42 -04:00
bvandeusen ed715dcc23 feat(knowledge): note types, counts, new-note button, audit fixes
- Add note_type (note/person/place/list) selector + entity metadata fields
  (relationship, email, phone / address, hours) to NoteEditorView
- Pre-select type via ?type= query param from KnowledgeView new-note dropdown
- KnowledgeView: add split "New note / ▾" button with type dropdown
- KnowledgeView: show per-type counts on sidebar filter buttons (when > 1)
- Fix: filter-btn now flex layout so count badge aligns to right edge
- Fix: list_notes count_query was missing parent_id filter (inflated totals)
- Fix: PATCH /api/notes/:id now fires upsert_note_embedding (workspace autosave)
- Fix: get_knowledge_counts endpoint for per-type counts
- Fix: get_knowledge_tags was silently discarding note_type filter (double-stmt bug)
- Fix: NoteEditorView onMounted stray brace from edit session

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-04 12:53:09 -04:00
bvandeusen 738245af5c fix(chat): audit fixes — retention, rag_project_id, cleanup scheduler, tool rounds
- cleanup_old_conversations now excludes briefing conversations (was
  silently deleting briefing history after the retention window)
- list_conversations response now includes rag_project_id, matching the
  shape returned by the single-conversation GET endpoint
- create_conversation_from_article: removed duplicate async_session import
  (_session2 was a copy of the same import); consolidated into one
- MAX_TOOL_ROUNDS fixed from 5→6 to match the actual range(6) loop;
  loop updated to range(MAX_TOOL_ROUNDS) so the constant is accurate
- Chat retention cleanup moved from per-request (every GET /conversations)
  to a daily scheduled job in event_scheduler.py; route no longer runs
  a DB write on every read

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-04 12:30:41 -04:00
bvandeusen edfed6b5bb feat(events): recurring expansion, CalDAV pull sync, past search, reminders
- RRULE expansion: list_events now expands recurring events into
  individual occurrences within the query window using python-dateutil
- CalDAV pull sync: new caldav_sync.py + POST /api/events/sync route;
  imports remote events into the internal store by caldav_uid
- Past event search: search_events accepts include_past=true to search
  historical events; exposed in the LLM tool definition
- Internal reminders: migration 0037 adds reminder_minutes +
  reminder_sent_at columns; event_scheduler.py checks every 5 min and
  fires push notifications; CalDAV sync job runs hourly
- reminder_minutes now stored and returned in create/update routes + tools

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-04 12:15:37 -04:00
bvandeusen 358534efbf fix(events): audit pass — 7 correctness fixes across the events system
Backend:
- tools.py: apply UTC normalization to update_event datetime fields
  (matched create_event which already did this)
- events.py service: allow end_dt/recurrence/project_id to be cleared
  via update_event by permitting None for nullable fields
- events.py service: find_events_by_query now returns upcoming events
  first, falling back to past — prevents AI tools from mutating stale
  past events when a future match exists
- events.py service: list_events now uses overlap logic (start <= to
  AND end >= from) so multi-day events spanning the query boundary
  are included; previously only start_dt was checked

Frontend:
- ToolCallCard: fire fable:calendar-changed on created/updated/deleted
  so CalendarView refetches without requiring a manual page refresh
- KnowledgeView: replace raw apiGet('/api/events') with listEvents()
  client function; also fix today bar which was reading .events off a
  flat array (always empty) — now correctly receives EventEntry[]
- HomeView: use full ISO strings for event date range instead of naive
  UTC-midnight strings; deduplicate inline date math via _dateRange()

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-04 12:02:10 -04:00
bvandeusen 90afbec4c2 feat(knowledge): two-tier pagination — ID pre-fetch + content batch loading
Backend:
- GET /api/knowledge/ids: returns up to 100 note IDs cheaply (no body
  parsing), supports same filters as /api/knowledge, includes has_more
- GET /api/knowledge/batch?ids=...: fetches full items for given IDs in
  order; used by frontend to load content in controlled batches

Frontend (KnowledgeView):
- Fetch 100 IDs upfront, load first 50 as content on mount
- IntersectionObserver sentinel (root: null) triggers 24-item content
  batches as user scrolls
- Proactive ID refill when queue drops below 48 unloaded IDs
- fetchGen counter invalidates stale in-flight responses on filter reset
- IDs claimed before async fetch to prevent double-loading
- sentinelVisible ref drives post-load re-check when content doesn't
  push sentinel off screen

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-04 09:44:50 -04:00
bvandeusen 968e536d3a fix(events): normalize naive datetimes to UTC in HTTP route
All datetime parsing now uses _parse_dt() which adds UTC tzinfo when
none is present, matching the fix already applied in tools.py. This
prevents asyncpg errors when comparing naive datetimes against
TIMESTAMPTZ columns — the root cause of events not appearing in the
calendar view.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-04 00:00:43 -04:00
bvandeusen 3887cab66e fix: knowledge infinite scroll + list_events timezone handling
KnowledgeView: sentinel was OUTSIDE the card-grid div, making
IntersectionObserver (root: cardGridEl) never fire since the target must
be a descendant of the root. Moved sentinel inside card-grid with
grid-column:1/-1 + order:9999 so it spans all columns and sits at the
bottom. Fixed backup check to compare against container bounds not viewport.

tools.py list_events: apply same UTC normalization as create_event (treat
naive datetimes as UTC, handle Z suffix). Update tool description to
explicitly request full-day UTC ranges so the LLM doesn't send local time
without offsets, which caused the recall query to miss UTC-stored events.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-03 21:10:43 -04:00
bvandeusen d624d38412 fix: KnowledgeView infinite scroll root + calendar event refresh
- KnowledgeView: IntersectionObserver was watching the viewport instead
  of the card-grid scroll container, causing infinite scroll to stop
  loading after only ~29 items. Pass card-grid element as `root`.
- CalendarView: listen for 'fable:calendar-changed' custom event and
  call refetchEvents() so tool-created events appear without navigation.
- ChatPanel: dispatch 'fable:calendar-changed' when create_event,
  update_event, or delete_event tool calls succeed.
- tools.py: normalize naive datetimes to UTC before storing events so
  timezone comparisons in list_events queries are always consistent.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-03 19:28:33 -04:00
bvandeusen 22003788f5 fix(generation): compute num_ctx in run_assist_generation 2026-04-03 13:35:47 -04:00
bvandeusen ef55bcb560 feat(llm): adaptive num_ctx tiers + fix KV cache priming num_ctx mismatch
Adds pick_num_ctx() which selects the smallest context window tier
(8192, 16384, 32768) that fits the current messages with 25% headroom,
capped at OLLAMA_NUM_CTX. Threads num_ctx through generation_task.py so
every chat request uses the computed tier rather than a fixed 16384.

Fixes a critical cache miss bug: KV cache priming in app.py and
settings.py was sending requests without num_ctx, so Ollama sized the
cache at its model default (different from the 16384 real requests used),
forcing a full model reload on the first real user message. Both priming
sites now call pick_num_ctx() and pass the matching value.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-03 11:47:39 -04:00
bvandeusen a6888953dc perf(prompts): trim tool schema descriptions to reduce prompt token count
Removes verbose redundant text from tool descriptions and system prompt
guidance: multi-line recurrence_rule JSON examples, CAPS warnings that
duplicate system prompt instructions, and wordy descriptions that don't
add model understanding.

Saves ~990 tokens per request (~17% reduction, 5,639 → ~4,650 tokens),
reducing prefill time on cache misses and lowering KV memory pressure.
No functional changes — parameter names, types, enums, and required
fields are unchanged.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-03 11:01:05 -04:00
bvandeusen c9065c4481 perf(settings): prime KV cache when user changes chat model
When a user saves a new default_model in Settings, fire a background
cache-prime request so the first message with the new model is fast
rather than paying the full cold-start prefill cost.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-03 10:51:17 -04:00
bvandeusen 4792e6459b perf(startup): prime Ollama KV cache with system prompt on warm-up
After loading each user's chat model into VRAM, send a minimal chat
request with the real system prompt (num_predict=1) to populate the
KV cache. The first real user message then only needs to process its
own tokens rather than the full ~5,600-token system prompt, reducing
cold-start TTFT from ~25s to <1s.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-03 10:47:59 -04:00
bvandeusen 3bd0dc6879 feat(settings): add background model picker with KV cache performance warning
Exposes OLLAMA_BACKGROUND_MODEL as a per-user setting in General settings,
alongside the Chat Model selector. Includes an inline warning when the same
model is selected for both, explaining the KV cache performance impact.

All background task callers (title generation, tag suggestions, project
summaries, RSS classification) now read background_model from user settings,
falling back to OLLAMA_BACKGROUND_MODEL env var.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-03 10:44:08 -04:00
bvandeusen fa38978745 fix(lint): remove unused model variable and get_setting import in chat.py
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-03 10:17:35 -04:00
bvandeusen 750a91898a perf(llm): route background tasks to dedicated model to preserve KV cache
Background tasks (title generation, tag suggestions, project summaries,
RSS classification) were using qwen3:8b and wiping its KV cache after
every response, preventing prefix cache hits on subsequent user messages.

Adds OLLAMA_BACKGROUND_MODEL (default: qwen2.5:0.5b) config var and
routes all background LLM calls to it, keeping qwen3:8b's KV cache
warm between user messages for consistent sub-second TTFT.

Also adds infinite scroll to KnowledgeView (replaces load-more button)
and bakes spaCy en_core_web_sm into the Docker image to eliminate the
pip install on every startup.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-03 01:33:54 -04:00