Phase 8 deleted the Python models for these tables; this migration
drops the orphan SQL.
Dropped tables (CASCADE-safe):
conversations, messages, generation_tool_log,
moments + moment_embeddings + moment_people/places/tasks/notes,
pending_curator_actions, push_subscriptions, weather_cache,
rss_item_embeddings (legacy pre-pivot experiment)
Dropped per-user settings: every voice_*, journal_*, briefing_*,
curator_* key, plus default_model, background_model, assistant_name,
auto_consolidate_tasks, chat_retention_days, think_enabled,
rag_default_scope.
Hard cutover — no downgrade. Existing data in these tables is lost;
the spec explicitly accepted this in exchange for a clean schema.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Tested that services/tools/_registry exposed event tools to the LLM
tool layer. That layer was removed in Phase 8 commit 91bafb6; event
CRUD is now covered via the MCP tools in test_mcp_tool_events.py.
The bench-*.md files got swept into the Phase 8 mega-commit by
`git add -A`. They were local working-tree notes the user never
intended to track.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The earlier sed-delete of the Push/ChatHistory/About sections from
the Notifications tab also clipped the tab's outer </div>. Vue's
type-checker happily accepted the unbalanced structure (templates
type-check on script bindings, not tag pairing) but Vite's Vue
compiler failed at build time:
Element is missing end tag.
file: src/views/SettingsView.vue:1062:5
(The reported line 1062 was the outermost .settings-content div —
Vue's parser blames the outermost open tag when an inner sibling
goes unclosed.)
Confirmed by counting: 115 open <div, 116 </div> before — and now
116/116 after restoring the closing tag between the Email Notifications
section and the Integrations tab.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Phase 6 smoke caught:
Error executing tool list_events:
can't compare offset-naive and offset-aware datetimes
Event.start_dt is stored timezone-aware; the wrapper was passing naive
datetimes built from datetime.fromisoformat("YYYY-MM-DD"), so the SQL
comparison crashed. Also: the docstring promises "date_to inclusive at
end-of-day" but the code was using midnight-of-date_to, which would
silently miss same-day events after midnight.
Extracted the range math into _day_range_utc() so create/update_event's
_combine() can stay as-is (it stays naive — the service localizes
create/update inputs against the user's tz, that path didn't crash).
Test updated to match: assert tz-aware UTC datetimes and the +24h
bump for end-of-day-inclusive semantics.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
FastMCP defaults to an allow-list of localhost variants for the Host
header (DNS-rebinding protection). Any deployment behind a reverse
proxy hitting a non-localhost hostname (e.g. devassistant.traefik.internal)
gets 421 Misdirected Request with:
WARNING mcp.server.transport_security: Invalid Host header: <name>
The protection exists to stop a malicious browser page from rebinding
DNS to attack a localhost MCP server. Our deployment is HTTP transport
behind a reverse proxy with bearer-token auth, which already gates
every request — so the rebinding threat doesn't apply. Disabling
the check lets any Host through; auth still rejects unauthorized
requests at 401.
This also makes the integration test pass without test-only host
hackery — every realistic Host header (Traefik internal hostname,
CDN domain, custom DNS) now reaches FastMCP cleanly.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
FastMCP's transport_security module enforces a Host header as DNS-
rebinding protection. Raw-ASGI scope construction doesn't fill it in
automatically (real HTTP clients always send one), so the test
request was getting 421 Misdirected Request with a log warning:
Missing Host header in request
Production is unaffected — real curl, Claude Code, and any real
client send a Host header.
After fixing the /mcp path forwarding in 1fd303a, requests now reach
FastMCP — but its StreamableHTTPSessionManager raises:
RuntimeError: Task group is not initialized. Make sure to use run().
The session manager owns a task group that must be running before it
can handle requests. In a stand-alone Starlette app this happens via
the `lifespan` parameter (lifespan = session_manager.run). Hosted
inside Quart, my dispatch wrapper only forwards HTTP events, not
lifespan, so the manager never got its startup signal.
Fix: hook session_manager.run() (an async context manager) into
Quart's @app.before_serving and @app.after_serving so the task group
is alive across the serving window.
The CI integration test was hitting the same crash because it drives
app.asgi_app raw without going through Quart's serving lifecycle —
@before_serving never fires. Updated the test to manually enter
session_manager.run() around the request.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The dispatch wrapper was rewriting scope['path'] from '/mcp' to '/'
before handing off to FastMCP. But FastMCP's streamable_http_app
mounts the JSON-RPC handler at '/mcp' (its default), so the rewritten
'/' had no matching route and FastMCP returned 404. Auth middleware
was correctly firing first (a no-auth request still gets 401), the
bug was only on the post-auth path.
Symptom: `claude mcp add ...` succeeds, registration shows in
`claude mcp list`, but connection fails because the initialize
handshake returns 404 instead of an MCP capabilities response.
Fix: pass the scope through unmodified. FastMCP's own routing matches
the '/mcp' path.
Also tightened the integration test that should have caught this —
it was asserting `status != 401`, which a 404 trivially passes. Now
asserts `== 200`, the actual expected response for initialize.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The generated Claude Code snippet was outputting:
claude mcp add ... scribe-dev --url <URL> --header ...
But `claude mcp add` errors out with `unknown option '--url'`. The
URL is a positional argument, not a flag:
claude mcp add [--transport ...] [--scope ...] <name> <url> [--header ...]
Dropped --url and put the URL inline as a positional. Claude Desktop
JSON snippet was already correct (uses {url, headers} keys, not a
CLI flag).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
MCP clients see tools namespaced by the server's local name already
(mcp__<server>__<tool>), so the fable_ prefix on every tool name was
redundant and ate tokens in the model's tool list.
Tools renamed (34 total):
fable_search → search
fable_list_notes / get_note / create_note / update_note / delete_note → list_notes / ...
fable_list_tasks / get_task / create_task / update_task / add_task_log → list_tasks / ...
fable_list_projects / get_project / create_project / update_project → list_projects / ...
fable_list_milestones / create_milestone / update_milestone → list_milestones / ...
fable_list_events / create_event / get_event / update_event / delete_event → list_events / ...
fable_list_tags → list_tags
fable_get_recent → get_recent
fable_list_persons / create_person / update_person → list_persons / ...
fable_list_places / create_place / update_place → list_places / ...
fable_list_lists / create_list / update_list → list_lists / ...
Also rebranded in MCP scope:
FastMCP("fable", ...) → FastMCP("scribe", ...)
auth realm "fable-mcp" → "scribe-mcp"
ASGI scope key fable_user_id → scribe_user_id
ContextVar label fable_mcp_user_id → scribe_mcp_user_id
Tool docstrings "in Fable" / "Fable task" → "in Scribe" / "Scribe task"
Server _INSTRUCTIONS prose
Deliberately kept:
- The internal Python package name `fabledassistant` (per project naming
convention — internal stays).
- "Fabled Scribe" as the official product/brand name (page footer,
smtp_from_name default).
- References to the legacy `fable-mcp/` standalone package in docstrings
explaining what we ported from — accurate until that directory is
deleted in Phase 10.
Client impact: existing MCP registrations need
claude mcp remove <name> && claude mcp add ...
once with a freshly-copied snippet from Settings → MCP Access. Claude
Code then re-discovers tools on connect — old conversations that
referenced fable_* tool names will see "tool not found" on those calls
until updated.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Settings → MCP Access now lets you tune the generated snippet:
- Server name input (default: 'scribe', stored per browser in
localStorage). The name appears both in the claude mcp add
command and as the JSON key in claude_desktop_config.json's
mcpServers map.
- Scope dropdown: user / project / local. Drives the --scope
flag in the claude mcp add snippet. Picks 'project' to commit
the server into the current repo's .mcp.json.
User-visible 'Fable' → 'Scribe' in MCP Access tab copy (lead text,
Claude Desktop step). Branding pivot in the rest of the app
(assistant_name placeholder, SMTP defaults, version line, etc.) is
deferred — chat/journal copy is going away in Phase 7 anyway.
Deliberately NOT touched:
- Tool names (fable_*) — protocol-level identifiers; renaming
breaks any Claude session, agent, or automation that referenced
them. Warrants its own phase.
- mcp/server.py: FastMCP('fable', ...) server name — same reason.
- Internal package name fabledassistant — per the project's
naming convention (CLAUDE.md memory), internal stays.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Rewrites the apikeys settings tab for the new MCP architecture:
- Tab label: 'API Keys' → 'MCP Access'
- Shows the in-app MCP URL (<origin>/mcp) with a copy button
- Claude Code snippet uses --transport http + --url + --header
- Claude Desktop snippet uses {url, headers: {Authorization}}
- Drops the wheel-download flow, the 'Other' client tab, and the
stdio env file / Claude config download helpers — those were
for the standalone fable-mcp package which goes away in phase 8
The api_keys backend stays unchanged — keys double as bearer tokens
for the /mcp endpoint via the existing auth.py middleware.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Replaces the Ollama HTTP get_embedding with a fastembed.TextEmbedding
singleton loaded lazily on first call. Model: BAAI/bge-small-en-v1.5
(384-dim), cached to /data/fastembed-cache.
Public API unchanged:
- get_embedding(text, model=None) — `model` now silently ignored
- upsert_note_embedding
- semantic_search_notes
- backfill_note_embeddings
_cosine_similarity gains a defensive length-mismatch check so any
stale 768-dim row that survived the migration is treated as 0.0
similarity rather than crashing zip().
The Ollama client dep stays in pyproject for now (other services still
use it); Phase 7 removes it once chat/journal/curator are gone.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Nine tools — list/create/update for each of person, place, list.
Get and delete reuse fable_get_note / fable_delete_note (typed
entities share the Note model).
Lists: the wrappers accept an `items: list[str]` for ergonomics and
translate to the {text, checked} dict shape that
services/knowledge.py and KnowledgeView.vue expect. items=[] clears;
items=None leaves unchanged.
Updates do an explicit get → merge → update round trip so updating
one typed field doesn't clobber the others stored alongside it in
entity_meta (which is a single JSONB column).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two cross-type bootstrap tools:
- fable_list_tags: tag vocabulary with usage counts, top-N by count.
Aggregation in Python (not SQL UNNEST) — trivial perf cost at
personal scale, much easier to test.
- fable_get_recent: most-recently-touched items across notes, tasks,
projects, events. Useful for Claude to ask 'what was I working on
recently' at the start of a conversation.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Five new tools (events weren't in fable-mcp before). Split
start_date + start_time inputs combine into a naive datetime that
services/events.py interprets in the user's local timezone.
Sentinels for update:
- empty strings → leave unchanged
- duration_minutes=-1 → leave unchanged
- duration_minutes=0 → set to point event (NULL duration)
- start_date/start_time must BOTH be set to move the event
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Seven tools matching existing fable-mcp contracts:
- fable_list/get/create/update_project (no delete; archive via status)
- fable_list/create/update_milestone (no get; no delete)
LLM-era similarity-check / 'confirmed' guard for create_project is
NOT replicated — Claude doesn't need it. The service's auto-summary
regeneration side effect (services.projects.update_project) stays
for now; gets removed in Phase 7 along with all other LLM code.
Notable sentinels:
- update_milestone: order_index=-1 means "leave unchanged" (0 is valid)
- create_milestone: description="" becomes None at the service layer
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Five tools wrapping services/notes.py with is_task=True (tasks are
notes with non-null status) plus services/task_logs.create_log for
add_task_log. Matches existing fable-mcp contracts. No delete_task —
preserves existing surface; cancel by updating status to "cancelled".
fable_get_task enriches with parent_title (extra service call when
parent_id is set), matching the existing route's behavior.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Five tools wrapping services/notes.py with is_task=False. Signatures
mirror the existing fable-mcp note tool contracts so Claude usage is
unchanged.
Key behavior the tests pin down:
- list_notes repackages (rows, total) tuple into {notes, total}
- tag=""/search_text="" are "no filter" sentinels
- update_note ONLY sends non-default fields to the service (the
main risk: a default empty string overwriting real data)
- tags=[] is an explicit clear; tags=None is "leave unchanged"
- project_id=0 on create => orphan; on update => leave unchanged
(preserved limitation from existing fable-mcp)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Establishes the tool pattern: each tool module exposes register(mcp),
register_all() aggregates them, build_mcp_server() calls register_all.
fable_search mirrors the existing fable-mcp contract (q/content_type/limit
in; {results, total} out) but calls services.embeddings.semantic_search_notes
directly instead of going over HTTP. User comes from mcp.current_user_id().
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds mcp._context.current_user_id() backed by a ContextVar. The ASGI
auth middleware sets it before dispatching to FastMCP and resets it
on the way out, so tool handlers can read the acting user without
re-parsing the request scope.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Driving Quart's full request pipeline via a hand-rolled ASGI scope
(no lifespan startup, no hypercorn-provided state) doesn't produce
a response. The 3 remaining tests cover the actual MCP middleware
behavior. The bypass property is implicit — if the middleware ate
non-/mcp requests, every existing /api/* test would fail.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Quart's test_client expects its request pipeline to populate
app._preserved_context. Our /mcp middleware deliberately bypasses
that pipeline (forwarding straight to FastMCP), so test_client's
teardown blew up with AttributeError. The middleware is correct;
the test harness was wrong.
Build raw ASGI scope/receive/send and call app.asgi_app directly —
which is what production hypercorn does anyway.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Wires FastMCP's streamable-HTTP ASGI sub-app into the Quart app via
asgi_app replacement. Requests under /mcp are stripped, auth-checked
against api_keys, and forwarded to FastMCP with fable_user_id set on
the ASGI scope. All other paths pass through to the original Quart
dispatch unchanged.
Tests cover the three auth paths (no header, invalid token, valid
token) plus a regression check that non-/mcp paths bypass the MCP
dispatch.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Thin parser over the existing api_keys lookup. Strips the Bearer
prefix, validates the token via services/api_keys.lookup_key (which
already filters revoked keys and updates last_used_at), and returns
the user_id for the in-flight MCP request.
Tests follow the existing mock-async_session pattern in
test_api_keys.py rather than introducing a real DB fixture.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Empty FastMCP instance with the post-pivot instructions block. Tools
get registered in phases 2 and 3; ASGI mounting + bearer-auth comes
in task 1.4.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
First step of the MCP-first pivot. Adds the official Anthropic MCP SDK
so we can mount a FastMCP HTTP endpoint inside the main Quart app.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The previous diagnostic instrumentation only wrote to stdout — fine for
'tail the logs while debugging', useless for 'crash happened at 3am
and Docker rotated the logs by morning'. This commit makes the
diagnostic state durable across container restart, OOM-kill, and log
rotation by writing to the mounted /data volume.
Four artifacts in /data/diagnostics/:
- current.json — overwritten atomically every heartbeat. Holds the
last known good snapshot (rss, asyncio_tasks, db_pool, curator_busy,
uptime, pid). Post-crash, this file alone tells you what the app
was doing 0-60 seconds before it died. Atomic write (tmp+rename)
so a crash mid-write can't leave a half-written file.
- last_shutdown.json — written when SIGTERM/SIGINT is caught OR
after_serving fires cleanly. If this file's mtime is older than
current.json's, the previous run died WITHOUT calling shutdown
(== SIGKILL, OOM-kill, or container hard-stop).
- last_exception.json — written when the asyncio exception hook
fires. Includes task name, coro name, exception type and message
alongside the resource snapshot.
- diag.log + diag.log.1..5 — rotating file log (10 MB × 5 backups
= 50 MB cap) containing every heartbeat, signal, and exception.
Separate from the app's stdout logger so Docker log rotation
can't take it out.
- previous_run.json — written at startup IF the post-mortem detects
the previous run died abruptly. Includes the abrupt-death snapshot
preserved for retrospection, so a recurring crash pattern can be
diffed over time.
Post-mortem at startup:
- Reads current.json + last_shutdown.json mtimes.
- If current.json is newer (== no clean shutdown happened after the
last heartbeat), logs a WARNING: 'PREVIOUS RUN DIED ABRUPTLY. Last
heartbeat was Xs before this startup. Last-known state: {...}'
- The warning lands in BOTH stdout AND the persistent diag.log, so
the operator notices it even if they only check one place.
- Stashes the abrupt-death snapshot in previous_run.json for later.
How the operator uses this after a crash:
1. cat /data/diagnostics/current.json -- last known good state
2. cat /data/diagnostics/last_shutdown.json -- did it shut down cleanly?
3. cat /data/diagnostics/last_exception.json -- any unhandled exception?
4. tail -100 /data/diagnostics/diag.log -- the lead-up
If current is newer than last_shutdown and last_exception doesn't
exist: SIGKILL or OOM (uncatchable). Check docker exit code 137
and host dmesg for oom-killer lines.
If last_exception.json exists: a background task crashed. The
traceback in the file names the coro.
If current.json's rss_mb was climbing across heartbeats: memory
leak / OOM trajectory. Bound the cause to whatever was active.
If current.json's db_pool checked_out was climbing: connection leak.
Look for code paths opening async_session() without exiting
'async with'.
If curator_busy=true across multiple heartbeats: curator hung on
Ollama. Restart Ollama or the Scribe stack to release the lock.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Recurring app/db crashes with no clear cause in existing logs.
Adds three crash-class indicators with minimal overhead (~1 log
line/min, 0.1ms work per heartbeat).
services/diagnostics.py:
1. **Heartbeat** every 60s logs a snapshot:
- RSS memory (from /proc/self/status — no deps).
- asyncio task count.
- DB pool: size / checked_in / checked_out / overflow.
- Curator busy state (from is_curator_running()).
- Uptime.
A sudden silence in heartbeats bounds the crash time to within
60s. The last snapshot before silence usually rules in or out:
memory growth -> OOM, pool exhaustion -> connection leak, hung
curator -> stuck async task.
2. **Signal handler** for SIGTERM/SIGINT logs the signal name +
final snapshot before letting Hypercorn handle the actual
shutdown. Distinguishes 'orderly shutdown via signal X' from
'silent log gap then container exit code 137' (SIGKILL / OOM-kill
are uncatchable; their absence in our log IS the diagnostic).
3. **Asyncio exception hook** logs full tracebacks for unhandled
task exceptions with the task/coro name. Default behaviour
swallows these silently — exactly the pattern that locked us
out of chat at 409 for an hour back on 2026-05-22 before we
added the guard around run_generation.
app.py wires start_diagnostics() into before_serving and
stop_diagnostics() into after_serving. stop_diagnostics emits one
final snapshot so the silence that follows is intentional, not a
crash.
How to use the new logs to diagnose:
- App restarts with 'received SIGTERM' in the last lines:
Orderly shutdown (docker stop / swarm restart / manual). Look
upstream for who issued it.
- App restarts with no shutdown line, last heartbeat 30+s before:
Likely SIGKILL — OOM-kill or container resource limit. Check
'docker ps -a' for exit code 137, or 'dmesg | grep -i kill' on host.
- App restarts with no shutdown line, heartbeat showed climbing
RSS: Memory leak. Snapshot the last heartbeat's MB value vs
earlier — if it doubled over hours, OOM is the cause.
- App restarts, db_pool checked_out kept growing: Connection leak.
Look for code paths that open async_session() but never exit
the 'async with' block.
- App seemed alive but stopped responding to requests, heartbeats
continued: Curator hung holding _CURATOR_RUN_LOCK. Check
curator_busy=true across multiple heartbeats — if stuck >5min,
the Ollama call hung. Restart Ollama or the Scribe stack.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Three coordinated changes per operator request 2026-05-24:
1. Settings UI rename matching the language we actually use:
- Chat Model -> Chat & Voice Model
- Worker Model -> Curator Model
Setting KEYS (default_model / background_model) unchanged on
purpose; renaming them requires a migration touching 50+ call
sites for purely UX-facing benefit.
2. Settings UI help text rewritten:
- Chat & Voice: documents that it handles chat AND small
conversational automations (titles, tags). Recommends
OLLAMA_NUM_PARALLEL=2+ on the Ollama server so background
automations get their own KV-cache slot and don't evict
the chat model's working state.
- Curator: notes the app enforces SERIAL execution regardless
of NUM_PARALLEL — only one curator pass runs at a time. This
matters most for 70b CPU models where a second instance
would waste system RAM.
3. Enforce serial curator execution globally:
- New module-level _CURATOR_RUN_LOCK in services/curator.py.
- run_curator_for_conversation now wraps its body in 'async
with _CURATOR_RUN_LOCK' — every entry point (scheduler sweep,
manual route trigger, future hooks) is serialized through it.
- is_curator_running() helper exposes the lock state.
- routes/journal.py manual trigger checks is_curator_running()
first and returns 409 {busy: true} immediately rather than
blocking the HTTP request for minutes waiting for a 70b CPU
pass to finish. The user can retry once the curator clears.
Why a 409 instead of queue: a curator pass on a 70b CPU model
can take 5+ minutes. Tying up an HTTP worker that long is bad;
making the user wait without feedback is worse. 409 surfaces
the busy state immediately and the user retries when they want.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Three prompt fixes addressing real failure modes observed in dev
journal data (conv 312, May 23):
curator.py — JOURNAL_CALIBRATION:
1. Strengthen the one-call-per-beat rule. Previous wording said 'do
not collapse multiple beats' but didn't explicitly forbid the
reverse: multiple record_moment calls for the SAME beat with
different phrasings. Observed in moments 7+8, 9+10, 11+14, 12+15,
13+16 — same content captured twice within a single curator pass.
New rule: explicit 'EXACTLY ONE tool call per distinct beat', plus
a 'check whether you already recorded this beat this turn' step.
2. Rewrite the save_person/save_place guidance. Previous wording
over-emphasized 'better to skip than invent' to the point that
the curator ignored explicit user introductions like 'my father's
name is Dale and my mother's name is Lynn, we went to Olive Garden'
— no save_person for Dale or Lynn, no save_place for Olive Garden.
The conservative-skip rule should apply to AMBIGUOUS mentions
('a friend told me'), not to explicit introductions. New rule
spells this out with positive examples.
journal_prep.py — _PREP_SYSTEM_PROMPT:
Extend the no-invent guards. The existing rule covered weather
specifically; today's prep added new fabrications:
- 'tasks due today include X' when tasks_due_today is empty and X is
actually 64 days overdue
- 'at 1:00 PM' when no time exists in the data
- 'currently in progress' applied to tasks where status is 'todo'
Three new rules: (a) never invent a task's due status — frame by the
bucket it actually appears under; (b) never invent times of day —
tasks have dates, not times; (c) never paraphrase a task's status
to something the data doesn't say.
journal_pipeline.py — JOURNAL_CALIBRATION:
1. Promote the one-question rule from buried bullet to top of the
prompt, with stronger phrasing ('ONE question per reply, MAXIMUM
... if you find yourself writing a second question mark, delete
it'). Observed: 3 questions per reply in every conv 312 assistant
turn ('how was it? what'd you order? did she enjoy it?').
2. Add explicit no-fishing rule: don't ask the user to share pictures,
send details, fetch information for the model. Reacts to what they
actually said, not what they didn't. Observed: 'do you have any
pictures you can share?' on msg 789.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Chat and background model roles effectively swapped during the
conversation+curator pivot, but call sites still used OLD routing.
This commit re-routes each call to the model whose new role fits.
Moved to background_model (worker — heavy, deliberate):
- services/journal_prep.py: daily prep generation.
- services/user_profile.py: observation consolidation.
Moved to default_model (chat — small, fast):
- services/chat.py save_response_as_note: note title generation.
- services/tag_suggestions.py: tag suggestions.
Already routed correctly (unchanged): curator, closeout, consolidation,
project summaries, history summarization.
SettingsView.vue: help text rewritten for both model fields to
describe new roles. Background Model UI label renamed to Worker
Model so the heavier role is visible from the picker. Warning copy
updated to recommend OLLAMA_MAX_LOADED_MODELS=2+ so chat and worker
can stay loaded simultaneously.
Schema names default_model and background_model unchanged on purpose
(renaming requires migration + touches ~50 call sites for UX-only gain).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The frontend half of the review queue. Closes the curator approval
loop end-to-end.
JournalView.vue:
- New 'Needs Review' section in the right rail, ABOVE the Captures
panel (per the design decision: pending stands out, captures are
ambient). Hidden entirely when nothing is pending so the rail stays
calm.
- Each pending action renders as a card:
- Header: action_type chip (e.g. 'update_note') + human-readable
title built from pendingTitle() ('Update Famous Supply network
restage', 'Delete Old grocery list', etc.).
- Diff body:
- For deletes: a red 'Permanent delete' warning.
- For updates: field-level diff rows (field name | old | → | new)
computed by pendingDiff(), which compares the curator's payload
against the snapshot taken at proposal time. Skips lookup-only
params (query, task, project, milestone, confirmed) so the diff
shows only what'd actually change.
- Empty-diff fallback for tools without snapshot helpers.
- Approve / Reject buttons. Disabled while a request is in flight
via reviewingIds Set so double-clicks can't fire twice.
- Approve calls approvePendingAction → server replays the original
tool call with authority='user'; toast on success/error.
- Reject calls rejectPendingAction → marks rejected, no execution.
- Both actions refresh the pending list AND the moments list (since
approving an update_note could affect what shows in captures).
- loadPendingActions() also runs after every manual curator trigger
and on initial mount, so the panel reflects current state without
manual page refresh.
CSS: warm-tinted panel using --color-warning so the section visually
distinguishes from the neutral captures feed below. Approve button
in success-green, reject in muted. Diff rows use a grid layout with
old-value strikethrough and an arrow separator.
End-to-end demo loop:
1. Have a journal conversation that includes 'mark the Famous Supply
task as done'.
2. Wait for curator sweep or hit 'Process captures'.
3. Curator search_notes('Famous Supply'), then update_note(...) is
intercepted by execute_tool(authority='curator') and queued.
4. The Needs Review panel shows: 'Update task Famous Supply network
restage' with status diff todo→done.
5. Click Approve → execute_tool replays with authority='user' →
the task moves to done. Card disappears from Needs Review.
This is the last C* commit in the queue. The curator now has a safe
path to mutate user data via proposals, with the user firmly in the
loop on every change.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The HTTP surface for the review queue. Three endpoints, all under
the existing /api/journal blueprint to keep the journal-related routes
together:
- GET /api/journal/pending — list current user's pending actions.
- POST /api/journal/pending/<id>/approve — replay the proposed tool
call via execute_tool(authority='user'). On success, marks
the row 'approved'; on replay error, leaves it pending so
the user can retry.
- POST /api/journal/pending/<id>/reject — marks 'rejected' with no
execution.
Each route is a thin wrapper around services/pending_actions and
delegates user-scoping to the service (which checks user_id on every
load — actions are private to the proposer).
api/client.ts:
- PendingCuratorAction interface mirroring the backend dict shape:
id, user_id, conv_id, action_type, target_type/id/label, payload,
current_snapshot, status, timestamps.
- listPendingActions / approvePendingAction / rejectPendingAction
helpers for the upcoming Needs Review panel.
C5 next: the panel itself.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The interceptor that closes the loop on the curator review queue.
With this commit, the curator can call update_note / update_milestone
/ update_project / update_profile / delete_note — those calls are
caught by execute_tool's authority='curator' path, snapshotted, and
written to pending_curator_actions for the user to approve or reject
later. Additive tools still run immediately.
services/tools/_registry.py:
- New _CURATOR_MUTATING_TOOLS frozenset: {update_note, update_milestone,
update_project, update_profile, delete_note}. update_event /
delete_event intentionally excluded — calendar events should always
be explicit user intent.
- execute_tool gains a keyword-only parameter, defaulting
to 'user'. Default behaviour is unchanged; existing callers keep
working without changes.
- When authority='curator' AND tool is in _CURATOR_MUTATING_TOOLS,
_queue_for_review captures a snapshot of the target via a per-tool
helper and writes a pending action. Returns {success:true,
pending:true, action_id:N, message:...} so the curator sees the
call as 'completed' for its bookkeeping.
- Per-tool snapshot helpers: _snapshot_note (covers update_note +
delete_note — uses the same fuzzy match update_note_tool uses, so
the snapshot reflects what'd actually be mutated), _snapshot_milestone,
_snapshot_project, _snapshot_profile. Snapshot capture is best-effort
— failure logs but still queues with empty snapshot so a curator
proposal never silently drops.
services/curator.py:
- Allowlist now includes the five mutating tools. They're safe to expose
because execute_tool intercepts them; the curator can propose without
being able to actually mutate.
- The execute_tool call now passes authority='curator'.
- System prompt explicitly authorizes the proposal pattern:
'update_note', 'update_milestone', 'update_project', 'update_profile',
'delete_note' are described as proposing tools that wait for user
approval. 'Don't try to update or delete anything' line removed.
services/pending_actions.py:
- approve() now passes authority='user' on the replay so the curator
interceptor doesn't re-route the replay back into pending and create
an infinite loop.
What's left in the queue:
- C4: API routes (list/approve/reject endpoints).
- C5: Frontend Needs Review panel.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The backend foundation for curator-proposed mutations awaiting user
approval. No tools route to this yet — that's C3's job. This commit
just lands the schema and the service API everything else will use.
Migration 0051 — new table:
- id, user_id (CASCADE), conv_id (SET NULL — survives conv deletion).
- action_type (the tool name to replay), target_type/target_id/
target_label (display hints).
- payload (jsonb — the curator's proposed args, replayed verbatim
on approval).
- current_snapshot (jsonb — the target's state at proposal time, so
the review UI can render an honest diff even if other work modified
the entity between proposal and review).
- status ('pending' / 'approved' / 'rejected') + CHECK constraint.
- created_at / reviewed_at.
- Partial index ix_pending_curator_actions_user_pending narrowed to
status='pending' — the Needs Review panel hits this constantly,
history rows just accumulate.
Model: PendingCuratorAction with to_dict() for API serialization.
Service services/pending_actions.py:
- create_pending(...) — called from the curator interceptor (C3).
Accepts an already-fetched current_snapshot so each mutating tool
can capture target state in its own way (notes vs milestones vs
profile have different shapes).
- list_pending(user_id, limit=50) — what the Needs Review panel reads.
- approve(action_id, user_id) — replays via execute_tool and marks
approved on success. Stays pending on replay error so the user
can retry. NOTE: approve passes the request through execute_tool
unchanged for now; C3 will add authority='user' so the upcoming
curator interceptor doesn't re-intercept the replay and loop.
- reject(action_id, user_id) — marks rejected with no execution.
C3 next: wires the curator interceptor (authority='curator' on
execute_tool routes mutating tools to create_pending instead of
running them), adds the mutating tools back to the curator's
allowlist, and updates approve() to pass authority='user'.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Layer 2 of the surfacing strategy (per 2026-05-23 design discussion).
The curator already has search_notes / search_journal / search_projects
in its allowlist for entity resolution; this commit just directs it
to use those searches more broadly — to surface relevant past work
that connects to today's beats.
Specifically, the system prompt now instructs the curator to:
- Search for projects/topics/people the user mentions, even when not
strictly needed for record_moment entity linking.
- Weave 1-2 short references to relevant past entries into the final
summary line, when they connect meaningfully to today's beats.
The summary feeds back into the chat model's system prompt on the
next turn (per Phase 3 of the architecture), so the chat model gains
contextual awareness of related past work without needing tools to
retrieve it itself.
Light explicit guardrails in the prompt: don't enumerate (avoid 'found
5 related notes'), don't invent references (only mention what was
actually retrieved), don't force a connection when nothing relevant
turns up.
This is the prompt-only Layer 2. Layer 1 (always-on RAG injection
into chat context) was already in place. Layer 3 (dedicated 'you
might want to revisit' surface in the right rail) is deliberately
deferred until 1+2 are observed in practice.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two related tightenings to the curator's behavior, both driven by user
questions about scope (2026-05-23):
1. **Tighten the prompt to extract beats only from User: lines.**
The transcript shows each message prefixed with role (User: / Assistant:).
The previous prompt instructed the model to capture beats but didn't
explicitly forbid using Assistant: content as a source. A small or
medium model could read 'It sounds like you had coffee with Sarah'
from an Assistant: line and turn it into a moment, even though that's
the assistant paraphrasing the user — not a user statement.
New prompt explicitly: Only User: lines are journal entries. Assistant:
lines are context for disambiguation only. Never create a record from
content that appears only in Assistant: text.
2. **Additive-only tool allowlist for the curator.**
The curator previously had access to the full journal tool set —
including update_*, delete_*, create_event, set_rag_scope, etc. The
architecture removed tools from the chat for exactly the reason that
confidently-wrong tool calls corrupt user data; the curator faces
the same risk async. Filtering the tool list at curator-time keeps
the boundary tight even if the system prompt fails to dissuade the
model from hallucinated tool names.
New _CURATOR_ALLOWED_TOOLS frozenset includes:
- Additive primary work: record_moment, create_note (handles both
notes and tasks via status), log_work (appends to existing task
timeline — additive on its own row), save_person, save_place,
create_project, create_milestone.
- Read-only helpers needed for entity resolution: search_notes,
search_projects, search_journal, list_tasks, list_projects,
list_milestones, read_note, get_project, get_profile.
Explicitly excluded: every update_*, every delete_*, create_event
(calendar events need explicit user intent, not curator inference),
set_rag_scope, lookup/research_topic/search_images (different
surface entirely).
Two-layer enforcement: the system prompt lists what's available and
forbids the rest, AND the actual tools list passed to Ollama is
filtered to the allowlist. So even if the model hallucinates a
forbidden tool name, the call can't fire — execute_tool returns
'Unknown tool: <name>'.
Bonus cleanup: _format_transcript now skips system and tool-role
messages. They were noise for the curator's task (system prompts
are instructions, tool results are JSON from prior calls). The
narrowed transcript matches the contract the prompt enforces.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
bvandeusen