Commit Graph

350 Commits

Author SHA1 Message Date
bvandeusen 5d40f2113f feat(tools): record_moment requires task_titles from prior search_notes
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-12 16:43:21 -04:00
bvandeusen 9a96fdb3c0 refactor(tools): resolve_project uses shared score_project_match helper
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-12 16:43:05 -04:00
bvandeusen 460959f0d4 refactor(tools): search_projects uses shared score_project_match helper
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-12 16:42:34 -04:00
bvandeusen 0dbbb98cf5 feat(tools): score_project_match helper with substring-first tiering
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-12 16:42:01 -04:00
bvandeusen 4b7ca1b17e feat(tools): search_notes description nudges away from type-nouns in query
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-12 16:41:28 -04:00
bvandeusen 65a3689aaa fix(notes): list_notes strips type-nouns from q before ILIKE filter
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-12 16:41:15 -04:00
bvandeusen 42c11dedae feat(notes): _strip_type_nouns helper for search query sanitization
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-12 16:40:54 -04:00
bvandeusen 0fbb1fbd92 fix(tools): search_projects returns success:True so result isn't mislabeled error
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-12 16:40:27 -04:00
bvandeusen 4e9eead3ab feat(profile): GET /api/profile/observations returns recent raw entries
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-12 14:36:42 -04:00
bvandeusen fc6ebf81eb feat(journal): closeout catch-up on startup when slot already passed
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-12 14:36:29 -04:00
bvandeusen b88d5ee6b3 feat(journal): register per-user closeout job at day_rollover_hour
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-12 14:35:57 -04:00
bvandeusen 4403026797 feat(journal): run_for_user orchestrates closeout extraction
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-12 14:34:42 -04:00
bvandeusen 552943d6c0 feat(journal): closeout system prompt with structured-field guard
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-12 14:33:57 -04:00
bvandeusen 2576be9e49 feat(journal): _build_transcript caps content and message window
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-12 14:33:27 -04:00
bvandeusen e17fc088b2 feat(journal): journal_closeout._filter_messages excludes daily_prep
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-12 14:33:03 -04:00
bvandeusen c5b0344240 feat(journal): add closeout_enabled to default config
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-12 14:32:40 -04:00
bvandeusen c33cab7020 fix(journal): wire weather refresh on config save; drop orphaned cache rows
Two related gaps in the journal weather panel:

1. Saving locations via PUT /journal/config didn't trigger a weather
   fetch, so newly-entered sites had no cache row (or a stale one) until
   the user manually clicked the panel's refresh button. The panel
   rendered "two sites with empty values" against pre-existing cache
   rows that no longer matched what the user had configured.

2. get_cached_weather_rows returned every WeatherCache row for the user
   regardless of whether the location was still in journal_config.
   Briefing-era rows survived migration 0040 (which only deleted the
   briefing_config setting, not the cache table) and showed up as
   ghost tabs in the UI.

Changes:
- get_cached_weather_rows accepts an optional valid_keys filter; rows
  whose location_key is not in the set are excluded.
- routes/journal.py:
  - put_config kicks off a background refresh_location_cache for any
    saved location with valid lat/lon.
  - GET /weather and POST /weather/refresh both pass valid_keys derived
    from the current config so orphaned rows don't surface.
- services/journal_prep.py filters the weather section to currently-
  configured locations as well; uses a lazy import of get_journal_config
  to avoid a cycle (journal_scheduler imports journal_prep).

153 tests pass.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-29 20:37:23 -04:00
bvandeusen 84640a0dc4 fix(events-tools): refuse ambiguous query, require event_id to disambiguate (#161)
Root cause of the 2026-04-29 dentist-appointment incident: the model
called update_event(query="Appointment") when two events had
"Appointment" in their titles. find_events_by_query returned both,
upcoming-first ordered by start_dt — matches[0] was id=2 (a stale
pre-existing event with garbage end_dt), not id=15 (the one the user
just created via the journal flow). update_event_tool silently took
matches[0] and mutated the wrong event.

Fix: a new resolver helper `_resolve_event_for_action` funnels both
update_event_tool and delete_event_tool through one disambiguation
path. Lookup precedence:
  - `event_id` → exact get_event lookup, no query at all
  - `query` matching exactly one event → proceed
  - `query` matching zero → return success=False, "no event found"
  - `query` matching 2+ events → return success=False with a
    `candidates` array of {id, title, start_dt, location} so the
    model can pick one and call again with `event_id`

The candidates list is capped at 8 to keep the model's context tight.
The error message names the count and the next-step ("pass event_id
or refine the query") so the model can self-correct in one turn.

For delete_event, the disambiguation is even more important — the
silent-matches[0] path would have deleted the wrong event outright
rather than just mutating it. The tool description leans into that:
"Deleting the wrong event is a costly user error; never guess."

Tool surface change: `query` and `event_id` are now both optional;
the tool errors clearly when neither is supplied. The model already
knows id values from prior tool results (returned in `data.id`),
which is the natural feeder for the disambiguation flow.

5 new tests in test_calendar_tool_tz.py cover:
- ambiguous query → success=False with candidate list, no mutation
- event_id supplied → bypasses query lookup entirely
- non-existent event_id → clear "no event found" error
- neither identifier → "query or event_id required" error
- same disambiguation enforced for delete_event_tool

46 calendar/events tests pass; ruff clean.

Closes Fable #161.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-29 14:29:13 -04:00
bvandeusen 4c58603009 feat(events): replace end_dt column with duration_minutes (#160)
Structural fix for the "end before start" bug class observed on prod
2026-04-29. Bad data became inexpressible at the schema level instead
of getting trapped in defensive read-path filters.

The hotfix that landed earlier today (94b169f) is reverted by the
preceding revert commit; this commit supersedes it cleanly with a
proper data-model change.

## Schema (migration 0043)

- Add `duration_minutes INTEGER NULLABLE` column on `events`.
- CHECK constraint: ``duration_minutes IS NULL OR duration_minutes >= 0``.
- Backfill from existing `end_dt`:
    - end_dt valid (end > start) → duration_minutes = total minutes
    - end_dt == start → duration_minutes = 0 (zero-duration point)
    - end_dt NULL or end_dt < start → duration_minutes = NULL
      (the corrupt prod row collapses cleanly to a point event)
- Drop the `end_dt` column. The wire format is preserved — `to_dict()`
  emits `end_dt` as a derived `start_dt + duration_minutes`. Existing
  API consumers (Flutter app, web frontend, CalDAV sync) keep
  receiving the same response shape; they just no longer have a way
  to PUT a stored `end_dt` that disagrees with `start_dt`.

## Service layer

- `Event.end_dt` becomes a `@property`. Setting it would require a
  setter we deliberately don't define — writes always go through
  `duration_minutes`.
- `_normalize_duration` is the single source-of-truth for input
  reduction. Accepts (start, end_dt, duration_minutes), returns the
  canonical `duration_minutes`, raises `ValueError` for negative
  durations, end-before-start, or end/duration disagreement.
- `create_event` and `update_event` accept either `end_dt` or
  `duration_minutes` for ergonomic compat; both convert via
  `_normalize_duration`. Update validates the post-update state when
  the patch includes either.
- `list_events` filter is simpler now: a coarse SQL prefilter
  (`start_dt <= date_to`) plus Python-side refinement using the
  derived `end_dt`. Avoids Postgres-specific interval arithmetic in
  the WHERE clause; refinement runs over a per-user result set so
  there's no scan-cost concern at personal scale.
- Recurring-event expansion uses `event.duration_minutes` directly
  instead of computing `end - start`. No more negative-timedelta
  hazard.

## CalDAV sync (incoming + outgoing)

- `caldav_sync.py` (pull) and `calendar_sync.py` (Radicale upsert)
  both convert iCal `DTEND` → `duration_minutes` on the way in.
  Outbound iCal still emits `DTEND` as `start_dt + duration_minutes`
  via the model's derived property. iCal interop is unchanged.

## Behavioral upgrade for `update_event`

Pure end_dt model: moving start past the existing end_dt would either
silently corrupt or hard-reject. Duration model: the duration is
preserved by default, so moving start slides the effective end
forward — which is what users mean when they "move" an event.
Explicit clear is still possible via `end_dt=None`.

## Tests

`tests/test_events_service.py`:
- 6 new `_normalize_duration` unit tests (sugar conversion, zero
  duration valid as point event, end-before-start rejected, negative
  duration rejected, inconsistent end+duration rejected, none → None)
- New behavioral test: `update_event` preserves duration when only
  start_dt changes (sliding semantics)
- New: clearing `end_dt=None` on update collapses to point event
- New: list_events surfaces a point event in the upcoming window
- New: list_events excludes a timed event whose effective end has
  already passed
- Existing mock-event helper updated to use `duration_minutes`
  instead of stored `end_dt`.

44 event-related tests pass; ruff clean.

## Out of scope (separate task)

Fable #161 — `find_events_by_query` returning multiple matches and
silently picking matches[0]. The exact root cause of how event id=2
got mutated in the first place; orthogonal to the storage model.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-29 14:19:44 -04:00
bvandeusen b7e7073425 Revert "fix(events): tolerate corrupt end_dt + reject end<=start at write time"
This reverts commit 94b169f31c.
2026-04-29 14:18:01 -04:00
bvandeusen 94b169f31c fix(events): tolerate corrupt end_dt + reject end<=start at write time
A prod event surfaced today with `start_dt=2026-05-01T12:00Z` and
`end_dt=2026-03-30T12:00Z` — end was 32 days BEFORE start, almost
certainly from an earlier tool-call mishap (Fable #161). The
list_events filter trusted the bogus end_dt and excluded the event
from every read path that hit the upcoming window, even though
start_dt was correctly in range. The event stayed visible in the
calendar grid (different range) but vanished from "Upcoming",
search, briefings, and journal prep events list.

This is the hotfix half of the response. The structural follow-up is
Fable #160 — replace end_dt with a duration column so invalid state
becomes inexpressible.

## A. Filter robustness in list_events

Treat `end_dt <= start_dt` as if no end_dt exists. The filter now
splits into two branches:
- valid duration: end_dt IS NOT NULL AND end_dt > start_dt AND
  end_dt >= date_from
- no/invalid duration: (end_dt IS NULL OR end_dt <= start_dt) AND
  start_dt >= date_from

Same change applied to the recurring-event expansion's `duration`
calculation, which was producing negative timedeltas for corrupted
rows and computing nonsensical occurrence end times.

## B. Write-side validation in create/update

`create_event` and `update_event` now raise ValueError when the
resulting state would have end_dt <= start_dt. Update validates
against the *post-update* state, not just the field being changed —
so pushing start_dt past an existing end_dt also fails loudly. Bad
data shouldn't be persistable from any write path.

Surfaced cleanly:
- Calendar tool wrappers (create_event_tool / update_event_tool)
  catch ValueError and return `{success: false, error: ...}`, which
  the model can read and self-correct.
- Route handlers (POST /api/events, PATCH /api/events/<id>) catch
  and return HTTP 400 with the validator's message instead of
  letting it bubble to a 500.

4 new tests in test_events_service.py:
- create rejects end before start
- create rejects equal start/end (zero duration)
- update validates the post-update state (start pushed past existing end)
- list_events surfaces an event whose end_dt is before its start_dt

34 event-related tests pass; ruff clean.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-29 13:48:28 -04:00
bvandeusen 35fab6cbf7 fix(calendar-tool): forbid placeholder events in create_event description
Reproducer (2026-04-29 dentist appointment): user said "this Friday,
I have an appointment" with no other details. The model immediately
called create_event with title="Appointment", description="User
mentioned an appointment this Friday but hasn't provided details
yet.", all_day=true. THEN it asked the user for time/location in
its reply. When the user came back with "8am at my dentist for
permanent crown fitting", the model called update_event — but never
updated the title, leaving the placeholder "Appointment" in the
calendar permanently.

The bug isn't about the tool surface, it's that the model created
an event before it had real content. The system prompt had no rule
against this, so the model hedged: "log a placeholder, ask for
details, then update". That pattern pollutes the calendar with
garbage titles and forces immediate update_event calls.

create_event tool description now includes an explicit anti-pattern:
record a moment, ask for the missing pieces, and only call create_event
once you have actual title + time + location. Stand-in titles like
"Appointment" / "Meeting" / "Event" with "details TBD" descriptions
are explicitly named as the failure mode.

Pure prompt change. 18 tests pass; ruff clean.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-29 10:54:39 -04:00
bvandeusen 9f8b451d15 fix(journal-prep): bucket tasks + drop non-proximate events (#159)
Two filtering issues that made the daily prep noisy and trained the
user to ignore it.

## Tasks: bucket into due-today / upcoming / overdue

The prep was calling `list_notes(due_before=day_date)` and labeling the
result as "tasks due today". That filter is strictly less-than, so it
returned only OVERDUE tasks (a single 68-day-stale task in this user's
case), while the prompt still framed them as fresh today's work. Each
day of the prep treated the same overdue task as new — the user
learned to ignore the line entirely.

`gather_daily_sections` now runs three queries:
- `tasks_due_today` — `due_after=day_date AND due_before=day_date+1`
- `tasks_upcoming` — next 7 days, exclusive of today
- `tasks_overdue` — strictly before today
Overdue entries carry a `days_overdue` count. `_render_sections_for_prompt`
emits three labeled headers ("TASKS DUE TODAY", "UPCOMING TASKS",
"OVERDUE TASKS (still on the list, not currently due)"). The system
prompt has a new TASK BUCKETS rule telling the model: don't call
overdue items "due today"; surface them with their staleness duration
("still on the list 68 days") and frame as a backlog reminder rather
than today's work.

Backwards-compat: `sections["tasks"]` still exists, now as the union
of all three buckets — strictly more useful than the prior overdue-
only behavior any frontend consumer was getting before.

## Events: tz-aware window + proximity filter

The user's "Birthday — 2026-09-29 (FREQ=YEARLY)" event was surfacing
in every daily prep, 5 months out. Root cause: `gather_daily_sections`
built `day_start`/`day_end` as NAIVE datetimes; `list_events` then
called `rrulestr(...).between(naive_from, naive_to)` against an
aware `dtstart`, which throws TypeError, hits the `except Exception`
fallback, and appends the canonical event row — regardless of whether
today is anywhere near a recurrence.

Fix:
1. Construct the day window as TZ-aware in the user's local timezone
   and convert to UTC before the query. RRULE expansion now runs
   correctly.
2. Defense-in-depth `_filter_proximate_events` drops events whose
   start_dt is more than 7 days from `day_date` (in the user's local
   TZ — not UTC, so a Friday 23:00 NY event isn't misclassified as
   Saturday). If list_events ever leaks a far-future row again, the
   prep doesn't surface it.

10 new tests in `tests/test_journal_prep_filtering.py` cover task
bucketing (overdue marker, due-today no-marker, no-due-date), the
proximity filter (the 4/29 reproducer, in-window keeps, local-vs-UTC
boundary, unparseable dates kept rather than suppressed), and the
rendering (overdue staleness shown, due-today doesn't repeat the date,
correct section ordering).

53 tests pass across journal_prep + journal_search + record_moment +
calendar_tool + events. Ruff clean.

Closes Fable task #159.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-29 09:31:12 -04:00
bvandeusen 4f18023284 fix(journal): server-side guards on record_moment links + place names (#158)
Belt-and-suspenders to the prompt-layer changes in 6c309f1. Even when
the model emits bogus task or place links, the server now refuses to
persist them.

## Task auto-linking guard

Reproducer (2026-04-27): a moment about restaging Docker on the swarm
ended up with `task_ids: [2]` (Weston's ADHD Evaluation) — the only
task in that day's prep. The model picked it up as filler.

`_filter_task_ids_by_keyword_overlap` now runs after id resolution: it
fetches each linked task's title, tokenizes both content and title
through `_content_keywords` (lowercased, stopwords stripped, <3-char
tokens dropped), and drops any link whose title shares no meaningful
keyword with the moment content. The drop is logged at INFO so we can
observe how often it fires post-deploy.

The guard runs against the merged id list, so it covers both the
preferred `task_titles` resolution path and the discouraged explicit
`task_ids` path.

## Place placeholder guard

Reproducer (2026-04-27): `place_names=["work"]` got passed to
`record_moment`. "work" / "home" / "office" aren't places — they're
role-labels for already-known geocoded locations.

`_filter_placeholder_places` drops a small set of generic single-word
labels before name resolution. Real user-named places that happen to
be one word (e.g. "Akron") pass through.

## Tests

9 new unit tests in `tests/test_record_moment_guards.py` cover:
  - keyword tokenization & stopword stripping
  - placeholder place filtering (generic, case-insensitive, real-place
    pass-through)
  - keyword-overlap filtering (the exact 4/27 reproducer, the genuine-
    reference case, mixed/partial relevance, empty input)

13 tests pass; ruff clean.

Closes Fable task #158.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-29 09:25:16 -04:00
bvandeusen 6c309f1331 fix(journal): tune persona — capture-first, anti-overhelp, first-person moments (#157)
Three related rough edges in the journal voice surfaced from real
journal usage 2026-04-27 → 2026-04-29:

1. **Persona overhelps.** When the user logged "today I'm prepping for
   an ISP migration at Branch 14 Bedford for work at famous supply",
   the assistant came back with "ISP migrations can be tricky. Are you
   handling the network configuration yourself, or is there a team
   supporting you? Also, are there any specific tasks or checks you
   need to complete before the switch?" — pushing IT-helpdesk advice
   the user didn't ask for. The user had to push back. JOURNAL_PERSONA
   now leads with "CAPTURE first, advise only if asked" and the
   RESPONSE STYLE block has an explicit anti-pattern banning
   troubleshooting / checklist / process-advice follow-ups unless the
   user explicitly invites them.

2. **Moments stored in third-person observer voice.** The dentist
   appointment beat got written as "The user mentioned having an
   appointment this Friday but hasn't provided details yet." — reads
   like an LLM transcript annotation, not a journal jot. The
   record_moment tool's `content` description previously said "in the
   user's voice or third-person", which was the literal source of the
   bug. New phrasing requires first-person/imperative with concrete
   GOOD/BAD examples, and the JOURNAL_CALIBRATION block reinforces it.

3. **Inconsistent emoji use.** 4/27 was clinical, 4/29 had 😊 and 🛠️
   in the appointment confirmation. RESPONSE STYLE now bans emojis
   outright — the journal is a thinking-companion surface and the
   emoji warmth reads as out-of-register chat-bot tone.

Bonus while in here:
- New MOMENT ENTITY LINKING section explicitly forbids attaching a
  task_titles link unless the user references the task by name (the
  4/27 Docker→ADHD auto-link bug; rest of that fix is in #158).
- Same section rejects generic place placeholders ("work" / "home" /
  "office") in favor of letting the user name the real place.

22 tests pass (4 journal + 18 calendar tool); ruff clean.

Closes Fable task #157.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-29 09:21:35 -04:00
bvandeusen 03d725ea3e fix(calendar-tool): anchor today's weekday in prompts + verify expected_weekday on create/update
A user asked Fable to schedule "this Friday at 8am" on Wednesday 4/29
2026. The model picked 4/30 (Thursday) and confidently labeled it
"Friday." The TZ pipeline did everything correctly given the model's
date — the bug was upstream: the model was guessing weekdays from ISO
dates without an anchor, and the calendar tools had no way to verify.

Three layered fixes:

1. **System prompts now name the weekday alongside the ISO date.**
   Both the journal-conversation prompt and the general chat prompt
   used to say "Today is 2026-04-29 (America/New_York)." They now say
   "Today is Wednesday, 2026-04-29 (...)." LLMs are unreliable at
   deriving weekday names from ISO dates; supplying the name removes
   the guess.

2. **`expected_weekday` parameter on create_event / update_event.**
   When the model passes `expected_weekday="friday"`, the backend
   computes the resolved start_date's weekday in the user's local
   timezone and rejects mismatches with a self-correcting error
   ("Date 2026-04-30 falls on Thursday, not Friday. Recompute..."),
   without creating the event. The check is local-aware: a Friday
   23:00 event in Tokyo crosses midnight UTC but the local view
   stays Friday, and the validator respects that.

3. **Tool descriptions instruct echo-and-confirm.** create_event and
   update_event descriptions now tell the model: when the user names
   a weekday, state the resolved date in the reply BEFORE calling
   the tool, and pass `expected_weekday`. Costs nothing in code,
   reinforces the validator.

6 new tests — match success, mismatch rejection (with create/update
not invoked), omitted-param backcompat, invalid weekday name, local-
not-UTC weekday computation, and the update_event variant. All 18
calendar-tool tests + 33 event-related tests pass; ruff clean.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-29 08:43:32 -04:00
bvandeusen 611c940527 fix(calendar-tool): split start/end into date+time to make event creation TZ-durable
A user reported "next Friday at 8am" landing on the wrong day. The
current `start` parameter accepts a combined ISO datetime string — when
the model emits something like `"2026-05-01T00:00:00Z"`, the parser
correctly honors the UTC tag and stores `2026-05-01 00:00 UTC`, which
displays as `2026-04-30 19:00` for a UTC-5 user. The bug isn't in our
parser; it's that we let the model TZ-tag the calendar day at all.

The fix moves the foot-gun: `create_event` and `update_event` now
prefer split fields (`start_date` + `start_time`, plus end variants).
A `YYYY-MM-DD` string carries no TZ metadata for a model to mis-tag,
and the backend builds the local datetime explicitly via
`datetime.combine(date, time, tzinfo=user_tz).astimezone(UTC)`. Strict
regex validation rejects anything with a TZ suffix on either field.

The legacy combined `start` / `end` fields are kept as a fallback so
saved tool-call payloads in conversation history still replay; new
calls are steered toward the split shape via the tool description.

7 new regression tests cover Eastern, Pacific, Tokyo (positive offset),
all-day inference, strict-shape rejection on both fields, backcompat
with the legacy `start` field, and the same fix for `update_event`.
27 of the event-related tests pass; ruff clean.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-29 08:16:25 -04:00
bvandeusen 4cfea784a9 fix(projects): tab counter NaN when a status bucket is empty
services/projects.py:get_project_summary built task_counts dynamically
from a GROUP BY query, so a project with no done tasks would omit the
'done' key entirely. Frontend's TypeScript interface declares all three
lifecycle keys as required, and ProjectView.vue summed them to render
the Tasks tab counter — undefined + N = NaN.

Two fixes:
1. Backend: initialise task_counts with {todo: 0, in_progress: 0,
   done: 0} so the service returns the contract its consumers expect.
   Catches the same problem for HomeView's project widget and any
   other consumer.
2. Frontend: defensive ?? 0 on the tab-counter sum, so the existing
   deploy renders correctly even before the backend rolls.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-28 07:40:21 -04:00
bvandeusen de4b1d7c7e fix(weather): match prep behavior — serve cached weather regardless of age
The /api/journal/weather route was filtering out cache rows older than 24
hours via parse_weather_card_data, while journal_prep.py read the same
rows raw without freshness checking. Result: the daily prep referenced
"home" and "work" temperatures while the right-rail UI showed nothing —
two surfaces, same backing data, inconsistent visibility.

Two changes:

1. parse_weather_card_data no longer returns None for stale data.
   WeatherCard already exposes fetched_at and gracefully hides
   today_high / forecast fields when they're absent, so old data renders
   with whatever fields the cached forecast still covers.

2. The /weather route opportunistically schedules a background refresh
   for any cache row older than 4 hours. If the user's journal_config
   has lat/lon for that location_key, the refresh runs and the next
   page load gets fresh data; if no usable config, the refresh is a
   silent no-op and the stale cache is still served.

This makes prep and UI consistent. It also self-heals over time — once
locations are configured, stale caches get refreshed on the next page
load instead of waiting indefinitely.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-27 21:19:57 -04:00
bvandeusen 4faaa5246b fix(journal): make record_moment mandatory, not a "nice to have"
Inspection showed only ONE record_moment call across the entire day's
journal — and that one had hallucinated person_ids. Multiple clear beats
went uncaptured: AP installation, going to watch a show with daughter,
decompressing-with-game.

The prior calibration said "use record_moment freely for meaningful
beats" — too soft. The model treated it as optional, especially when
already in chatbot-reply mode.

Rewritten: record_moment is now framed as the model's PRIMARY JOB. The
calibration includes an explicit checklist of what counts as a beat
(event, encounter, decision, observation, plan, feeling, accomplishment)
and an explicit instruction to call record_moment FIRST, before composing
the reply. Multiple beats → multiple calls. The ONLY skip case spelled
out: purely meta-conversational messages (acknowledgements, meta-asks
about prior tool results).

Tests on a fresh conversation will tell us if this moves the needle —
today's journal is poisoned by ten prior chatbot-flavored turns that
the model is pattern-matching against in its own history.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-26 21:37:29 -04:00
bvandeusen 0ed9cbf666 fix(journal): restore prep prose; soften persona toward chat-like
Per user clarification: previous over-rotation dropped the LLM-generated
prep prose entirely (just a phase greeting) and made the chat persona
extremely sparse ("you are a place where words go down"). User actually
wanted only the chat replies pulled back, NOT the prep dropped, and the
chat to behave largely like normal /chat — asking follow-ups and
verifying earlier details.

services/journal_prep.py — restored:
- _render_sections_for_prompt
- _PREP_SYSTEM_PROMPT (the direct, briefing-style prompt from 590a07b)
- _generate_prep_prose
- _fallback_prep_text
- ensure_daily_prep_message now calls _generate_prep_prose again
- removed _phase_for_now / _phase_prompt helpers (no longer needed)

services/journal_pipeline.py — persona rewritten:
- Old: "You are the user's journal. Be quiet. Listen. You are not helpful."
- New: "You are the user's assistant. Behave like the rest of the app's
  chat: respond conversationally, ask follow-up questions, verify details
  from earlier turns, use tools naturally."
- Calibration block reorganized: PEOPLE/PLACES (ask first), MOMENTS
  (silent + use *_names), STATE-CHANGING TOOLS (confirmation flow),
  OTHER, RESPONSE STYLE.
- RESPONSE STYLE keeps the no-apologizing / no-option-menus /
  no-verbatim-repetition / match-user-length rules but drops the "be
  quiet, one short sentence" framing.

Net behavior:
- Open journal → LLM-generated prep prose with today's tasks/events/weather
- Reply → assistant responds conversationally like /chat, asks follow-ups,
  verifies details, uses tools
- Background: silently records moments via *_names, asks before creating
  new people/places

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-26 18:04:34 -04:00
bvandeusen 4668c0950b fix(journal): minimal prep + quieter persona
The prep was generating a multi-sentence recap of tasks/events/weather/
projects/recent moments via an LLM call. Per user direction, that's
redundant — the right-side widgets already show today's data — and the
verbosity made the journal feel chatty when the user wanted quiet.

Replaces the prep prose generator with a single phase-aware check-in
question (drawn from a static map: morning="How are you starting the
day?", midday="How's it going so far?", evening="How did the day shake
out?"). No LLM call. The structured `sections` are still gathered and
persisted on msg_metadata for provenance and possible future tooling
(e.g., search), they just don't render in the prep message.

Also pulls the journal persona way back. The prior framing pushed the
model toward stock therapy-template patterns ("I'm sorry you're feeling…"
+ numbered option lists). The new persona is "you are the user's journal —
listen, be quiet, stay out of the way." RESPONSE STYLE rules now lead the
calibration block and explicitly forbid:

- apologizing for the user's feelings
- offering to help / pitching tools
- multi-option menus
- verbatim repetition of prior replies
- padding short replies into paragraphs

Most replies should be one short sentence. Sometimes the right reply is
"Got it" + a record_moment tool call.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-26 17:48:41 -04:00
bvandeusen b728acd841 fix(journal): name-based entity resolution + tighter calibration + anti-repetition
Three real bugs surfaced from inspecting today's journal turns:

1. record_moment was getting fed hallucinated person_ids (the LLM passed
   [1, 2] instead of the IDs save_person had just returned). Result: the
   moment was linked to two random old test-data notes ("test task 2",
   "Tell a joke"), not the people the user actually mentioned.

2. The calibration rule "ask before save_person" was being silently
   ignored — model just called save_person on first mention of Victoria
   and Mother without asking the user.

3. The model produced a verbatim-identical reply to its previous turn when
   the user mentioned "overwhelmed" twice — same numbered-list of 4
   options, same closing line. The "warm listener / ask gentle questions"
   persona was pushing toward stock therapy-template patterns.

Fixes:

services/tools/journal.py — record_moment now accepts *_names parameters
(person_names, place_names, task_titles, note_titles). Server resolves
each name to a note ID via case-insensitive title match, scoped by
note_type or task-status. *_ids parameters still exist but are now
documented as DISCOURAGED. The LLM physically cannot invent the wrong ID
when using names — names with no match are silently dropped. Resolution
happens via _resolve_entity_ids_by_name helper.

services/journal_pipeline.py — JOURNAL_PERSONA tightened (no more
"warm/curious listener" framing that pushed toward stock comfort
patterns). JOURNAL_CALIBRATION rewritten as scannable sections with
imperative language: PEOPLE/PLACES require asking before save_person;
TASK/NOTE state changes use the confirmation flow; MOMENTS are silent
but MUST use *_names not *_ids; OTHER notes the no-set_rag_scope and
no-auto-notes invariants. Added a RESPONSE STYLE section that explicitly
forbids verbatim repetition and stock multi-option menus.

After deploy, force-regenerate today's prep via fable_trigger_journal_prep
to also pick up the tighter prep prompt from 590a07b.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-26 17:16:33 -04:00
bvandeusen 590a07bc13 fix(journal): tighten prep prompt — direct briefing, not flowery letter
Previous system prompt asked for "warm, conversational, like a friend
writing a letter" which produced flowery preludes that buried the actual
data. Rewritten to:

- Lead with practical data (tasks, events, weather) — concrete and specific
- 4-7 sentences total, tight prose, no padding
- Recent moments / open threads mentioned briefly at the END as context,
  not as the lead
- Voice: "competent assistant briefing the user" not "friend writing a letter"
- Close with a short journal invitation under 8 words

Also dropped max_tokens 600 -> 400 to bias toward concision.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-26 16:10:57 -04:00
bvandeusen c9f2134ad4 feat(journal): conversational LLM-generated daily prep (replaces card)
The structured prep card was data-rich but voiceless. Replaced with an
LLM-generated conversational opener — same shape the briefing's compilation
slot had — that renders as a normal assistant chat bubble at the top of
the day's conversation.

Backend (services/journal_prep.py):
- Renamed generate_daily_prep -> gather_daily_sections (still pure data
  fetching, no LLM); kept the old name as a backwards-compat alias.
- New _generate_prep_prose: hands the gathered sections to generate_completion
  with a warm-conversational system prompt; returns prose. Falls back to a
  plain greeting if the LLM call fails or no model is configured.
- ensure_daily_prep_message now persists the prep as role='assistant' with
  the prose as content. Structured sections stay on msg_metadata for
  provenance. Auto-upgrades legacy system-role preps in place on next call.

Frontend:
- Drop the <article class="daily-prep"> structured block from JournalView.
  The prep is now just the first chat bubble — picks up the existing
  Illuminated Transcript styling automatically.
- Drop dayMessages / prepMessage / prepSections / asArray helpers — no
  longer needed.
- ChatMessage hideMessage filter: comment refined to clarify it only
  catches LEGACY system-role prep rows. Current preps are assistant-role
  and render normally.

Net effect: open /journal -> first thing you see is a warm assistant bubble
that talks about your day -> input bar below to reply.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-26 15:42:30 -04:00
bvandeusen dbd9f00061 refactor: hard-cut RSS infrastructure (scope C)
Removes the entire RSS feature surface — feeds, items, embeddings, reactions,
discussion-note flow, briefing news context, settings, env-vars, and DB
tables. Keeps the URL-generic article-reader (the read_article LLM tool)
under a clean module so the LLM can still fetch arbitrary article content
from URLs the user provides.

Backend:
- New services/article_fetcher.py — single source of trafilatura URL→text
- New services/tools/article.py — read_article tool (was nested under tools/rss)
- Delete services/rss.py, rss_classifier.py, rss_filtering.py, article_context.py
- Delete services/tools/rss.py
- Delete models/rss_feed.py (RssFeed, RssItem), models/rss_item_embedding.py
- services/embeddings.py: drop upsert/semantic_search/backfill RSS helpers
- services/llm.py: remove _build_briefing_article_context, briefing-conv branch,
  ARTICLE_DISCUSS_SEED skip-RAG branch; drop get_rss_items / add_rss_feed from
  the actions list
- services/generation_task.py: drop _maybe_save_article_discussion_note + caller
- routes/chat.py: drop /api/chat/from-article/<id> endpoint
- routes/journal.py: re-import via web.py refactor (article_fetcher path)
- services/tools/__init__.py: register `article`, drop `rss`
- services/tools/_registry.py: drop the requires=='rss' check
- app.py: drop backfill_rss_item_embeddings + backfill_rss_article_content tasks
- config.py: prose-only edit (no env var change — RSS env vars were never first-class)

Frontend:
- stores/settings.ts: drop rssEnabled
- SettingsView.vue: drop the RSS-classification mention
- api/client.ts: drop openArticleInChat (the from-article endpoint is gone)

Tests:
- Delete tests/test_rss_service.py, test_news_api.py, test_article_reading.py

Migration:
- 0042_drop_rss: DROP TABLE rss_item_embeddings, rss_item_reactions, rss_items,
  rss_feeds; DELETE settings rows for rss_enabled / briefing_*_topics

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-26 12:33:30 -04:00
bvandeusen cacfcac86a fix(journal): restore weather + events panels; hide daily-prep system msg in chat
The journal UI was over-stripped earlier — weather panel, current-conditions
poll, and the upcoming-events sidebar were all dropped. Restored those (calls
the new /api/journal/weather, /api/journal/weather/current,
/api/journal/weather/refresh, /api/journal/weather/geocode endpoints).

Also: the daily-prep system message was rendering as flat text inside
ChatPanel because there's no .role-system bubble styling. Added a hideMessage
guard in ChatMessage so daily-prep system messages don't render in the chat
stream — they're already shown above as a structured prep card.

News / RSS reactions / article-discuss are intentionally NOT in the journal
(scoped out per user direction). The broader RSS infrastructure cleanup is
a separate, larger task.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-26 12:22:51 -04:00
bvandeusen ce76a003f7 feat(journal): /api/journal/* routes blueprint + cosine helper unit tests
Endpoints:
- GET/PUT /api/journal/config — per-user journal config
- GET /api/journal/today — today's journal conversation, generates daily prep on demand
- GET /api/journal/day/<iso_date> — past day's journal
- GET /api/journal/days — list of dates with journal content
- POST /api/journal/trigger-prep — manual regeneration of prep
- GET /api/journal/moments — list/search moments with filters
- PATCH /api/journal/moments/<id> — edit content/tags/pinned + junctions
- DELETE /api/journal/moments/<id>

Blueprint registered in app.py.
tests/test_journal_search.py — cosine helper coverage.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-25 22:40:56 -04:00
bvandeusen ac188c40a5 feat(journal): LLM tools (record_moment, search_journal) + system prompt wiring
- services/tools/journal.py — record_moment + search_journal tool handlers
- services/tools/_registry.py: add `journal` flag on ToolDef + tool() decorator
- get_tools_for_user(user_id, conversation_type='chat'|'journal') —
  exclude journal-only tools from chat sessions; exclude set_rag_scope
  from journal sessions
- services/tools/__init__.py: register the new journal module; drop the
  unused get_briefing_tools export
- services/llm.py build_context: short-circuit for journal conversations,
  using journal_pipeline.build_journal_system_prompt and skipping all
  notes-RAG injection (preserves the journal/notes isolation invariant)
- services/generation_task.py: pass conversation_type into get_tools_for_user

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-25 22:39:42 -04:00
bvandeusen d9ab538ef4 feat(journal): backend services (moments, search, prep, scheduler, pipeline)
- services/moments.py — CRUD with embedding sync + entity-link helpers
- services/journal_search.py — three-mode search (temporal / entity / semantic)
  with notes-RAG isolation; cosine-similarity helper unit-tested
- services/journal_prep.py — gathers tasks/events/weather/news/projects/
  recent-moments/open-threads into a structured prep block
- services/journal_scheduler.py — per-user APScheduler cron for daily prep,
  follows the BackgroundScheduler + threadsafe-async pattern from event_scheduler
- services/journal_pipeline.py — system prompt (persona + calibration rules)
  with last-48h ambient moments injection
- app.py: wire journal scheduler start/stop hooks
- routes/settings.py: re-add live-reschedule on timezone change (now via
  journal_scheduler.update_user_schedule)

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-25 22:37:39 -04:00
bvandeusen 7602bf2293 feat(briefing): hard-cut tear-down
Backend:
- Delete briefing services (pipeline, scheduler, conversations, profile, tools)
- Delete routes/briefing.py + remove blueprint registration
- Move _get_temp_unit into services/weather.get_temp_unit (reads top-level temp_unit setting)
- Rename briefing_preferences.py → rss_filtering.py (functions are RSS-specific)
- Strip briefing scheduler hooks from app.py
- Strip briefing scheduler call from routes/settings.py
- Update test imports (test_rss_service, test_tz_helpers)

Frontend:
- Delete BriefingView, BriefingSetupWizard, BriefingToolStatusRow
- Strip /briefing route + nav links (AppHeader, KnowledgeView)
- Strip Settings → Briefing tab + state + functions + imports
- Strip briefing-intermediate handling from ChatMessage
- Hide /news route + nav links (NewsView depended on briefing endpoints; orphaned in tree)
- Drop unused useSettingsStore from AppHeader

The Android BriefingScreen lives in a separate repo and is not touched here.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-25 22:33:37 -04:00
bvandeusen d352e9264b feat(models): Moment + MomentEmbedding + junctions; rename Conversation.briefing_date → day_date
Also rename services/tz.user_briefing_date → user_day_date with a backwards
compat alias (briefing modules using the old name will be deleted in the
upcoming briefing tear-down stage). Update services/chat.py to_dict to use
day_date.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-25 20:40:02 -04:00
bvandeusen e652dece9b refactor: rename Fabled Assistant to Fabled Scribe
Updates user-visible branding across frontend (PWA manifest, page title,
Settings, push fallback), backend (email templates and subjects, LLM
system prompt, CalDAV displayname, SMTP from-name default), README,
quickstart compose, and MCP server description.

Also updates the CI image path and quickstart image reference to
git.fabledsword.com/bvandeusen/fabledscribe in preparation for the
Forgejo repo rename. Internal Python package, env vars, and database
schema unchanged.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-25 16:28:25 -04:00
bvandeusen 61e62a6904 fix: serialize article fetches in lookup to avoid lxml double-free
trafilatura.extract dispatched via run_in_executor isn't safe to run
concurrently — two parallel calls can crash the process with a
libxml2-level double free. The top-level Wikipedia+SearXNG gather is
fine; only the inner per-article extraction needs to stay sequential,
matching the pre-parallelization behavior.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-17 23:04:05 -04:00
bvandeusen 9a252c8dde feat: parallelize lookup (Wiki+SearXNG) and include Wikipedia thumbnails
Runs the Wikipedia summary and SearXNG search concurrently and returns
both when available, so current-event questions aren't masked by a
generic role article from Wikipedia. When the Wikipedia summary includes
a thumbnail, it is cached through the existing image pipeline and
surfaced as an embeddable markdown snippet alongside the extract.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-17 22:44:42 -04:00
bvandeusen 8db6b4d230 feat: add Wikipedia as research pipeline source alongside SearXNG
Runs wiki_search in parallel with SearXNG queries; Wikipedia results
(which already carry content via their extract field) are merged into
the source pool before outline generation, skipping a separate fetch
step. Also fixes a pre-existing F811 ruff violation in the test file.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-17 21:59:35 -04:00
bvandeusen d5e6a8f6da refactor: update all search_web references to lookup
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-17 21:57:46 -04:00
bvandeusen 06cd3493fd feat: replace search_web with unified lookup tool (Wikipedia + SearXNG fallback)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-17 21:56:31 -04:00
bvandeusen 6a8f0e9143 feat: add wikipedia service with summary lookup and search
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-17 21:54:44 -04:00
bvandeusen 619f069358 feat(weather): add refresh button and return card data from refresh endpoint
Add a refresh button (↻) to the weather section header in BriefingView so
users can manually re-fetch weather data (needed after deploying the hourly
precip changes to populate the cache). Update the existing POST
/api/briefing/weather/refresh endpoint to return full card data instead of
just location keys.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-17 16:40:57 -04:00