Commit Graph

16 Commits

Author SHA1 Message Date
bvandeusen 6c309f1331 fix(journal): tune persona — capture-first, anti-overhelp, first-person moments (#157)
Three related rough edges in the journal voice surfaced from real
journal usage 2026-04-27 → 2026-04-29:

1. **Persona overhelps.** When the user logged "today I'm prepping for
   an ISP migration at Branch 14 Bedford for work at famous supply",
   the assistant came back with "ISP migrations can be tricky. Are you
   handling the network configuration yourself, or is there a team
   supporting you? Also, are there any specific tasks or checks you
   need to complete before the switch?" — pushing IT-helpdesk advice
   the user didn't ask for. The user had to push back. JOURNAL_PERSONA
   now leads with "CAPTURE first, advise only if asked" and the
   RESPONSE STYLE block has an explicit anti-pattern banning
   troubleshooting / checklist / process-advice follow-ups unless the
   user explicitly invites them.

2. **Moments stored in third-person observer voice.** The dentist
   appointment beat got written as "The user mentioned having an
   appointment this Friday but hasn't provided details yet." — reads
   like an LLM transcript annotation, not a journal jot. The
   record_moment tool's `content` description previously said "in the
   user's voice or third-person", which was the literal source of the
   bug. New phrasing requires first-person/imperative with concrete
   GOOD/BAD examples, and the JOURNAL_CALIBRATION block reinforces it.

3. **Inconsistent emoji use.** 4/27 was clinical, 4/29 had 😊 and 🛠️
   in the appointment confirmation. RESPONSE STYLE now bans emojis
   outright — the journal is a thinking-companion surface and the
   emoji warmth reads as out-of-register chat-bot tone.

Bonus while in here:
- New MOMENT ENTITY LINKING section explicitly forbids attaching a
  task_titles link unless the user references the task by name (the
  4/27 Docker→ADHD auto-link bug; rest of that fix is in #158).
- Same section rejects generic place placeholders ("work" / "home" /
  "office") in favor of letting the user name the real place.

22 tests pass (4 journal + 18 calendar tool); ruff clean.

Closes Fable task #157.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-29 09:21:35 -04:00
bvandeusen 03d725ea3e fix(calendar-tool): anchor today's weekday in prompts + verify expected_weekday on create/update
A user asked Fable to schedule "this Friday at 8am" on Wednesday 4/29
2026. The model picked 4/30 (Thursday) and confidently labeled it
"Friday." The TZ pipeline did everything correctly given the model's
date — the bug was upstream: the model was guessing weekdays from ISO
dates without an anchor, and the calendar tools had no way to verify.

Three layered fixes:

1. **System prompts now name the weekday alongside the ISO date.**
   Both the journal-conversation prompt and the general chat prompt
   used to say "Today is 2026-04-29 (America/New_York)." They now say
   "Today is Wednesday, 2026-04-29 (...)." LLMs are unreliable at
   deriving weekday names from ISO dates; supplying the name removes
   the guess.

2. **`expected_weekday` parameter on create_event / update_event.**
   When the model passes `expected_weekday="friday"`, the backend
   computes the resolved start_date's weekday in the user's local
   timezone and rejects mismatches with a self-correcting error
   ("Date 2026-04-30 falls on Thursday, not Friday. Recompute..."),
   without creating the event. The check is local-aware: a Friday
   23:00 event in Tokyo crosses midnight UTC but the local view
   stays Friday, and the validator respects that.

3. **Tool descriptions instruct echo-and-confirm.** create_event and
   update_event descriptions now tell the model: when the user names
   a weekday, state the resolved date in the reply BEFORE calling
   the tool, and pass `expected_weekday`. Costs nothing in code,
   reinforces the validator.

6 new tests — match success, mismatch rejection (with create/update
not invoked), omitted-param backcompat, invalid weekday name, local-
not-UTC weekday computation, and the update_event variant. All 18
calendar-tool tests + 33 event-related tests pass; ruff clean.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-29 08:43:32 -04:00
bvandeusen 611c940527 fix(calendar-tool): split start/end into date+time to make event creation TZ-durable
A user reported "next Friday at 8am" landing on the wrong day. The
current `start` parameter accepts a combined ISO datetime string — when
the model emits something like `"2026-05-01T00:00:00Z"`, the parser
correctly honors the UTC tag and stores `2026-05-01 00:00 UTC`, which
displays as `2026-04-30 19:00` for a UTC-5 user. The bug isn't in our
parser; it's that we let the model TZ-tag the calendar day at all.

The fix moves the foot-gun: `create_event` and `update_event` now
prefer split fields (`start_date` + `start_time`, plus end variants).
A `YYYY-MM-DD` string carries no TZ metadata for a model to mis-tag,
and the backend builds the local datetime explicitly via
`datetime.combine(date, time, tzinfo=user_tz).astimezone(UTC)`. Strict
regex validation rejects anything with a TZ suffix on either field.

The legacy combined `start` / `end` fields are kept as a fallback so
saved tool-call payloads in conversation history still replay; new
calls are steered toward the split shape via the tool description.

7 new regression tests cover Eastern, Pacific, Tokyo (positive offset),
all-day inference, strict-shape rejection on both fields, backcompat
with the legacy `start` field, and the same fix for `update_event`.
27 of the event-related tests pass; ruff clean.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-29 08:16:25 -04:00
bvandeusen b728acd841 fix(journal): name-based entity resolution + tighter calibration + anti-repetition
Three real bugs surfaced from inspecting today's journal turns:

1. record_moment was getting fed hallucinated person_ids (the LLM passed
   [1, 2] instead of the IDs save_person had just returned). Result: the
   moment was linked to two random old test-data notes ("test task 2",
   "Tell a joke"), not the people the user actually mentioned.

2. The calibration rule "ask before save_person" was being silently
   ignored — model just called save_person on first mention of Victoria
   and Mother without asking the user.

3. The model produced a verbatim-identical reply to its previous turn when
   the user mentioned "overwhelmed" twice — same numbered-list of 4
   options, same closing line. The "warm listener / ask gentle questions"
   persona was pushing toward stock therapy-template patterns.

Fixes:

services/tools/journal.py — record_moment now accepts *_names parameters
(person_names, place_names, task_titles, note_titles). Server resolves
each name to a note ID via case-insensitive title match, scoped by
note_type or task-status. *_ids parameters still exist but are now
documented as DISCOURAGED. The LLM physically cannot invent the wrong ID
when using names — names with no match are silently dropped. Resolution
happens via _resolve_entity_ids_by_name helper.

services/journal_pipeline.py — JOURNAL_PERSONA tightened (no more
"warm/curious listener" framing that pushed toward stock comfort
patterns). JOURNAL_CALIBRATION rewritten as scannable sections with
imperative language: PEOPLE/PLACES require asking before save_person;
TASK/NOTE state changes use the confirmation flow; MOMENTS are silent
but MUST use *_names not *_ids; OTHER notes the no-set_rag_scope and
no-auto-notes invariants. Added a RESPONSE STYLE section that explicitly
forbids verbatim repetition and stock multi-option menus.

After deploy, force-regenerate today's prep via fable_trigger_journal_prep
to also pick up the tighter prep prompt from 590a07b.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-26 17:16:33 -04:00
bvandeusen dbd9f00061 refactor: hard-cut RSS infrastructure (scope C)
Removes the entire RSS feature surface — feeds, items, embeddings, reactions,
discussion-note flow, briefing news context, settings, env-vars, and DB
tables. Keeps the URL-generic article-reader (the read_article LLM tool)
under a clean module so the LLM can still fetch arbitrary article content
from URLs the user provides.

Backend:
- New services/article_fetcher.py — single source of trafilatura URL→text
- New services/tools/article.py — read_article tool (was nested under tools/rss)
- Delete services/rss.py, rss_classifier.py, rss_filtering.py, article_context.py
- Delete services/tools/rss.py
- Delete models/rss_feed.py (RssFeed, RssItem), models/rss_item_embedding.py
- services/embeddings.py: drop upsert/semantic_search/backfill RSS helpers
- services/llm.py: remove _build_briefing_article_context, briefing-conv branch,
  ARTICLE_DISCUSS_SEED skip-RAG branch; drop get_rss_items / add_rss_feed from
  the actions list
- services/generation_task.py: drop _maybe_save_article_discussion_note + caller
- routes/chat.py: drop /api/chat/from-article/<id> endpoint
- routes/journal.py: re-import via web.py refactor (article_fetcher path)
- services/tools/__init__.py: register `article`, drop `rss`
- services/tools/_registry.py: drop the requires=='rss' check
- app.py: drop backfill_rss_item_embeddings + backfill_rss_article_content tasks
- config.py: prose-only edit (no env var change — RSS env vars were never first-class)

Frontend:
- stores/settings.ts: drop rssEnabled
- SettingsView.vue: drop the RSS-classification mention
- api/client.ts: drop openArticleInChat (the from-article endpoint is gone)

Tests:
- Delete tests/test_rss_service.py, test_news_api.py, test_article_reading.py

Migration:
- 0042_drop_rss: DROP TABLE rss_item_embeddings, rss_item_reactions, rss_items,
  rss_feeds; DELETE settings rows for rss_enabled / briefing_*_topics

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-26 12:33:30 -04:00
bvandeusen ac188c40a5 feat(journal): LLM tools (record_moment, search_journal) + system prompt wiring
- services/tools/journal.py — record_moment + search_journal tool handlers
- services/tools/_registry.py: add `journal` flag on ToolDef + tool() decorator
- get_tools_for_user(user_id, conversation_type='chat'|'journal') —
  exclude journal-only tools from chat sessions; exclude set_rag_scope
  from journal sessions
- services/tools/__init__.py: register the new journal module; drop the
  unused get_briefing_tools export
- services/llm.py build_context: short-circuit for journal conversations,
  using journal_pipeline.build_journal_system_prompt and skipping all
  notes-RAG injection (preserves the journal/notes isolation invariant)
- services/generation_task.py: pass conversation_type into get_tools_for_user

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-25 22:39:42 -04:00
bvandeusen 7602bf2293 feat(briefing): hard-cut tear-down
Backend:
- Delete briefing services (pipeline, scheduler, conversations, profile, tools)
- Delete routes/briefing.py + remove blueprint registration
- Move _get_temp_unit into services/weather.get_temp_unit (reads top-level temp_unit setting)
- Rename briefing_preferences.py → rss_filtering.py (functions are RSS-specific)
- Strip briefing scheduler hooks from app.py
- Strip briefing scheduler call from routes/settings.py
- Update test imports (test_rss_service, test_tz_helpers)

Frontend:
- Delete BriefingView, BriefingSetupWizard, BriefingToolStatusRow
- Strip /briefing route + nav links (AppHeader, KnowledgeView)
- Strip Settings → Briefing tab + state + functions + imports
- Strip briefing-intermediate handling from ChatMessage
- Hide /news route + nav links (NewsView depended on briefing endpoints; orphaned in tree)
- Drop unused useSettingsStore from AppHeader

The Android BriefingScreen lives in a separate repo and is not touched here.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-25 22:33:37 -04:00
bvandeusen 61e62a6904 fix: serialize article fetches in lookup to avoid lxml double-free
trafilatura.extract dispatched via run_in_executor isn't safe to run
concurrently — two parallel calls can crash the process with a
libxml2-level double free. The top-level Wikipedia+SearXNG gather is
fine; only the inner per-article extraction needs to stay sequential,
matching the pre-parallelization behavior.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-17 23:04:05 -04:00
bvandeusen 9a252c8dde feat: parallelize lookup (Wiki+SearXNG) and include Wikipedia thumbnails
Runs the Wikipedia summary and SearXNG search concurrently and returns
both when available, so current-event questions aren't masked by a
generic role article from Wikipedia. When the Wikipedia summary includes
a thumbnail, it is cached through the existing image pipeline and
surfaced as an embeddable markdown snippet alongside the extract.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-17 22:44:42 -04:00
bvandeusen d5e6a8f6da refactor: update all search_web references to lookup
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-17 21:57:46 -04:00
bvandeusen 06cd3493fd feat: replace search_web with unified lookup tool (Wikipedia + SearXNG fallback)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-17 21:56:31 -04:00
bvandeusen 3ac32dc3bc feat: add rss_enabled user setting to toggle RSS functionality
RSS is now off by default. When disabled:
- Scheduler skips RSS feed sync during compilation slot
- Briefing pipeline skips RSS item gathering
- RSS LLM tools (get_rss_items, add_rss_feed) are hidden
- API routes return empty results for feeds/news
- Frontend hides News nav link, RSS Feeds and News Preferences in settings
- Briefing view hides news sidebar section

Toggle in Settings > Briefing > RSS / News.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-17 13:58:50 -04:00
bvandeusen e4e1d1da49 fix(tz): interpret calendar and briefing dates in user's local timezone
Two related bugs where the server defaulted naive datetimes to UTC instead
of the configured user timezone, causing all-day events to land on the
previous day and briefings to "disappear" at UTC midnight.

- New services/tz.py helpers: get_user_tz, user_today, user_briefing_date
  (the briefing day flips at 4am local to align with the compilation slot,
  so the 00:00-04:00 local window still shows yesterday's briefing until
  the new one is generated).
- calendar create/list/update tools now parse naive datetimes in the
  user's TZ before converting to UTC for storage, and tool descriptions
  tell the model to pass plain local dates.
- briefing_conversations.get_or_create_today_conversation and the
  reset-today route use user_briefing_date so the in-progress briefing
  doesn't get replaced at 19:00 NY / UTC midnight.
- _run_profile_closeout targets user-local "yesterday" for consistency.

Regression tests added for the TZ helpers and the calendar tool.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-13 15:35:27 -04:00
bvandeusen 0becc1439b fix(llm): correct context sizing, honor think requests, broaden delete
Three related fixes uncovered while benchmarking qwen3:14b against 8b:

- pick_num_ctx was only counting message content, missing the ~15K
  tokens of tool schemas. num_ctx=8192 was being selected while actual
  prompt_tokens hit 14K+, causing silent prompt truncation on every
  tool-using request. Now includes json.dumps(tools) in the estimate.
  KV cache priming in app.py and routes/settings.py also fetches tools
  so the primed num_ctx matches what real chat requests will use.

- _should_think's heuristic classifier was overriding explicit
  think=true requests from the frontend toggle and MCP, gating on
  message length and regex patterns. Now a pass-through — the caller
  is the source of truth. quick_capture hardcodes think=False since
  it's a fast classification path that was relying on the old gating.

- delete_note description only mentioned "note or task", so the model
  refused to call it for entries created by save_person / save_place /
  create_list. Description now explicitly lists all five note_types it
  handles.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-12 15:32:52 -04:00
bvandeusen 77339d5c58 refactor(tools): consolidate LLM tools from 42 to 38
Merge create_task into create_note (set status='todo' for tasks, omit
for notes), merge delete_task into delete_note, consolidate entity
tools (create/update_person → save_person, create/update_place →
save_place), rename get_note → read_note with clearer descriptions,
move calculate out of rag.py into utility.py, and extract shared
duplicate detection into check_duplicate() helper.

Updates all downstream references in generation_task.py, quick_capture.py,
ToolCallCard.vue, and WorkspaceView.vue.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-12 13:38:45 -04:00
bvandeusen ce2d76447c refactor(tools): decorator-based tool registry replaces monolithic tools.py
Split 2566-line tools.py into a tools/ package with @tool decorator
registration. Each tool's schema, metadata, and implementation live
together. Briefing eligibility is now a briefing=True flag instead of
a separate frozenset allowlist. Conditional inclusion (CalDAV, SearXNG)
uses requires= metadata. Public API (get_tools_for_user, execute_tool)
unchanged.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-12 10:01:00 -04:00