50 Releases 52 Tags

RSS Feed

• v26.05.12.1 76dc75a03b

  Compare

  v26.05.12.1 — Journal closeout + LLM tool-use fixes Stable

  bvandeusen released this 2026-05-12 21:32:01 -04:00 | 260 commits to main since this release

  Two threads — one rebuilds a half-deleted feature, the other patches an LLM-misuse failure mode caught in prod.

  Journal closeout for profile observations

  The briefing tear-down on 2026-04-25 deleted the closeout job but left the read path: build_profile_context still injects learned_summary into journal-prep and chat system prompts. Result was a stale frozen summary plus no path to refresh it. This release reinstates the write side with a hard guard against the briefing-era feedback loop.

  • services/journal_closeout.py (new) — runs once per user per day at the configured day_rollover_hour. Reads yesterday's /journal conversation, filters out messages with msg_metadata.kind="daily_prep" at the SQL layer, asks the background LLM for 2–5 bullet observations focused on user-side patterns (NOT structured fields like name/job/expertise — those are still the update_profile tool's lane), and appends to observations_raw via the existing append_observations.
  • Hard data-exclusion guard — the daily-prep block is structurally invisible to the closeout LLM. The previous briefing-era closeout sent all messages with role labels and relied on the prompt to say "user revealed"; that's the shape that caused the feedback loop. Instruction-only role separation isn't reliable on local Ollama models against long transcripts.
  • background_model for the LLM call — keeps chat-model KV cache warm.
  • closeout_enabled config key (default ON), wired to a Settings → Profile toggle. New GET /api/profile/observations endpoint and collapsible "Recent observations (14)" panel for transparency.
  • Catch-up on startup — if today's day_rollover_hour already passed and observations_raw has no entry for yesterday, runs once.

  LLM tool-use fixes from 2026-05-08 journal session

  Prod conversation #282 exposed three concrete code bugs and two behavioral patterns. All five fixes shipped together because they share a root failure mode: when the user names a project/task colloquially, the model fails to find it, fails to trust valid matches, or skips searching entirely and fabricates a moment.

  • search_projects returns success: True — the dispatcher reads this key to set status on the conversation history entry. Missing key → result labeled status: error even when data was returned → model second-guessed valid matches. One-line fix.
  • list_notes strips type-nouns from q — task, tasks, note, notes, project, projects are filtered out of the keyword-search tokens before the AND-ILIKE filter is built. Closed the case where "find the sebring secondary task" returned 7 unrelated tasks because task #95's title and body don't contain the word "task". search_notes tool description also tells the LLM to use type / project parameters instead of stuffing those words into query.
  • score_project_match shared helper (new in tools/_helpers.py) — tiered scoring (1.0 exact title / 0.85 substring either way / 0.70 query-in-description / SequenceMatcher-against-title fallback). Strips project / projects filler from the query so "famous supply project" substring-matches "Famous-Supply Work topics" at 0.85 instead of 0.228 SequenceMatcher floor. Both search_projects_tool and resolve_project use the helper — single source of truth, no drift between the LLM-facing surface and the create-task resolver.
  • record_moment description — task_titles / note_titles must come from a prior search_notes call in the same turn. Prevents the project-name-as-task-title misuse that triggered the original investigation.
  • Search-first heuristic added to both JOURNAL_CALIBRATION and the chat tool_lines static block: when the user references existing work, call search_notes first; create or record only after no match surfaces and the user confirms.

  Verification

  • 178+ backend pytest tests pass (CI)
  • tests/test_journal_closeout.py — filter, run_for_user happy + skip paths, scheduler register/disable, catch-up
  • tests/test_tool_use_fixes.py — _strip_type_nouns, score_project_match tiers, search_projects_tool ranks substring above SequenceMatcher, resolve_project finds colloquial match
  • Tool-use fixes verified end-to-end against fable-dev MCP: search_projects shows status: success, score 1.0 for exact-match substring; search_notes correctly split query="tool-use" / type="task" and returned the right task
  • Search-first heuristic verified in journal UI

  Deploy notes

  • No schema migrations; deploy-only
  • After deploy, optionally click Reset Learned Data in Settings → Profile to wipe the stale briefing-era learned_summary. The new closeout will rebuild from clean observations starting tomorrow's prep cycle.

  Out of scope (follow-ups)

  • Manual verification of the closeout toggle / observations panel (PR test plan checkbox left unchecked — covered separately)
  • Hybrid BM25 + embedding search (replaces semantic-only)
  • "Did you mean X?" UI surface for project matches in the 0.40–0.85 score band

  Downloads

  • Source Code (ZIP)
  • Source Code (TAR.GZ)