Tested that services/tools/_registry exposed event tools to the LLM
tool layer. That layer was removed in Phase 8 commit 91bafb6; event
CRUD is now covered via the MCP tools in test_mcp_tool_events.py.
Phase 6 smoke caught:
Error executing tool list_events:
can't compare offset-naive and offset-aware datetimes
Event.start_dt is stored timezone-aware; the wrapper was passing naive
datetimes built from datetime.fromisoformat("YYYY-MM-DD"), so the SQL
comparison crashed. Also: the docstring promises "date_to inclusive at
end-of-day" but the code was using midnight-of-date_to, which would
silently miss same-day events after midnight.
Extracted the range math into _day_range_utc() so create/update_event's
_combine() can stay as-is (it stays naive — the service localizes
create/update inputs against the user's tz, that path didn't crash).
Test updated to match: assert tz-aware UTC datetimes and the +24h
bump for end-of-day-inclusive semantics.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
FastMCP's transport_security module enforces a Host header as DNS-
rebinding protection. Raw-ASGI scope construction doesn't fill it in
automatically (real HTTP clients always send one), so the test
request was getting 421 Misdirected Request with a log warning:
Missing Host header in request
Production is unaffected — real curl, Claude Code, and any real
client send a Host header.
After fixing the /mcp path forwarding in 1fd303a, requests now reach
FastMCP — but its StreamableHTTPSessionManager raises:
RuntimeError: Task group is not initialized. Make sure to use run().
The session manager owns a task group that must be running before it
can handle requests. In a stand-alone Starlette app this happens via
the `lifespan` parameter (lifespan = session_manager.run). Hosted
inside Quart, my dispatch wrapper only forwards HTTP events, not
lifespan, so the manager never got its startup signal.
Fix: hook session_manager.run() (an async context manager) into
Quart's @app.before_serving and @app.after_serving so the task group
is alive across the serving window.
The CI integration test was hitting the same crash because it drives
app.asgi_app raw without going through Quart's serving lifecycle —
@before_serving never fires. Updated the test to manually enter
session_manager.run() around the request.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The dispatch wrapper was rewriting scope['path'] from '/mcp' to '/'
before handing off to FastMCP. But FastMCP's streamable_http_app
mounts the JSON-RPC handler at '/mcp' (its default), so the rewritten
'/' had no matching route and FastMCP returned 404. Auth middleware
was correctly firing first (a no-auth request still gets 401), the
bug was only on the post-auth path.
Symptom: `claude mcp add ...` succeeds, registration shows in
`claude mcp list`, but connection fails because the initialize
handshake returns 404 instead of an MCP capabilities response.
Fix: pass the scope through unmodified. FastMCP's own routing matches
the '/mcp' path.
Also tightened the integration test that should have caught this —
it was asserting `status != 401`, which a 404 trivially passes. Now
asserts `== 200`, the actual expected response for initialize.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
MCP clients see tools namespaced by the server's local name already
(mcp__<server>__<tool>), so the fable_ prefix on every tool name was
redundant and ate tokens in the model's tool list.
Tools renamed (34 total):
fable_search → search
fable_list_notes / get_note / create_note / update_note / delete_note → list_notes / ...
fable_list_tasks / get_task / create_task / update_task / add_task_log → list_tasks / ...
fable_list_projects / get_project / create_project / update_project → list_projects / ...
fable_list_milestones / create_milestone / update_milestone → list_milestones / ...
fable_list_events / create_event / get_event / update_event / delete_event → list_events / ...
fable_list_tags → list_tags
fable_get_recent → get_recent
fable_list_persons / create_person / update_person → list_persons / ...
fable_list_places / create_place / update_place → list_places / ...
fable_list_lists / create_list / update_list → list_lists / ...
Also rebranded in MCP scope:
FastMCP("fable", ...) → FastMCP("scribe", ...)
auth realm "fable-mcp" → "scribe-mcp"
ASGI scope key fable_user_id → scribe_user_id
ContextVar label fable_mcp_user_id → scribe_mcp_user_id
Tool docstrings "in Fable" / "Fable task" → "in Scribe" / "Scribe task"
Server _INSTRUCTIONS prose
Deliberately kept:
- The internal Python package name `fabledassistant` (per project naming
convention — internal stays).
- "Fabled Scribe" as the official product/brand name (page footer,
smtp_from_name default).
- References to the legacy `fable-mcp/` standalone package in docstrings
explaining what we ported from — accurate until that directory is
deleted in Phase 10.
Client impact: existing MCP registrations need
claude mcp remove <name> && claude mcp add ...
once with a freshly-copied snippet from Settings → MCP Access. Claude
Code then re-discovers tools on connect — old conversations that
referenced fable_* tool names will see "tool not found" on those calls
until updated.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Replaces the Ollama HTTP get_embedding with a fastembed.TextEmbedding
singleton loaded lazily on first call. Model: BAAI/bge-small-en-v1.5
(384-dim), cached to /data/fastembed-cache.
Public API unchanged:
- get_embedding(text, model=None) — `model` now silently ignored
- upsert_note_embedding
- semantic_search_notes
- backfill_note_embeddings
_cosine_similarity gains a defensive length-mismatch check so any
stale 768-dim row that survived the migration is treated as 0.0
similarity rather than crashing zip().
The Ollama client dep stays in pyproject for now (other services still
use it); Phase 7 removes it once chat/journal/curator are gone.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Nine tools — list/create/update for each of person, place, list.
Get and delete reuse fable_get_note / fable_delete_note (typed
entities share the Note model).
Lists: the wrappers accept an `items: list[str]` for ergonomics and
translate to the {text, checked} dict shape that
services/knowledge.py and KnowledgeView.vue expect. items=[] clears;
items=None leaves unchanged.
Updates do an explicit get → merge → update round trip so updating
one typed field doesn't clobber the others stored alongside it in
entity_meta (which is a single JSONB column).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two cross-type bootstrap tools:
- fable_list_tags: tag vocabulary with usage counts, top-N by count.
Aggregation in Python (not SQL UNNEST) — trivial perf cost at
personal scale, much easier to test.
- fable_get_recent: most-recently-touched items across notes, tasks,
projects, events. Useful for Claude to ask 'what was I working on
recently' at the start of a conversation.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Five new tools (events weren't in fable-mcp before). Split
start_date + start_time inputs combine into a naive datetime that
services/events.py interprets in the user's local timezone.
Sentinels for update:
- empty strings → leave unchanged
- duration_minutes=-1 → leave unchanged
- duration_minutes=0 → set to point event (NULL duration)
- start_date/start_time must BOTH be set to move the event
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Seven tools matching existing fable-mcp contracts:
- fable_list/get/create/update_project (no delete; archive via status)
- fable_list/create/update_milestone (no get; no delete)
LLM-era similarity-check / 'confirmed' guard for create_project is
NOT replicated — Claude doesn't need it. The service's auto-summary
regeneration side effect (services.projects.update_project) stays
for now; gets removed in Phase 7 along with all other LLM code.
Notable sentinels:
- update_milestone: order_index=-1 means "leave unchanged" (0 is valid)
- create_milestone: description="" becomes None at the service layer
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Five tools wrapping services/notes.py with is_task=True (tasks are
notes with non-null status) plus services/task_logs.create_log for
add_task_log. Matches existing fable-mcp contracts. No delete_task —
preserves existing surface; cancel by updating status to "cancelled".
fable_get_task enriches with parent_title (extra service call when
parent_id is set), matching the existing route's behavior.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Five tools wrapping services/notes.py with is_task=False. Signatures
mirror the existing fable-mcp note tool contracts so Claude usage is
unchanged.
Key behavior the tests pin down:
- list_notes repackages (rows, total) tuple into {notes, total}
- tag=""/search_text="" are "no filter" sentinels
- update_note ONLY sends non-default fields to the service (the
main risk: a default empty string overwriting real data)
- tags=[] is an explicit clear; tags=None is "leave unchanged"
- project_id=0 on create => orphan; on update => leave unchanged
(preserved limitation from existing fable-mcp)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Establishes the tool pattern: each tool module exposes register(mcp),
register_all() aggregates them, build_mcp_server() calls register_all.
fable_search mirrors the existing fable-mcp contract (q/content_type/limit
in; {results, total} out) but calls services.embeddings.semantic_search_notes
directly instead of going over HTTP. User comes from mcp.current_user_id().
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds mcp._context.current_user_id() backed by a ContextVar. The ASGI
auth middleware sets it before dispatching to FastMCP and resets it
on the way out, so tool handlers can read the acting user without
re-parsing the request scope.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Driving Quart's full request pipeline via a hand-rolled ASGI scope
(no lifespan startup, no hypercorn-provided state) doesn't produce
a response. The 3 remaining tests cover the actual MCP middleware
behavior. The bypass property is implicit — if the middleware ate
non-/mcp requests, every existing /api/* test would fail.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Quart's test_client expects its request pipeline to populate
app._preserved_context. Our /mcp middleware deliberately bypasses
that pipeline (forwarding straight to FastMCP), so test_client's
teardown blew up with AttributeError. The middleware is correct;
the test harness was wrong.
Build raw ASGI scope/receive/send and call app.asgi_app directly —
which is what production hypercorn does anyway.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Wires FastMCP's streamable-HTTP ASGI sub-app into the Quart app via
asgi_app replacement. Requests under /mcp are stripped, auth-checked
against api_keys, and forwarded to FastMCP with fable_user_id set on
the ASGI scope. All other paths pass through to the original Quart
dispatch unchanged.
Tests cover the three auth paths (no header, invalid token, valid
token) plus a regression check that non-/mcp paths bypass the MCP
dispatch.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Thin parser over the existing api_keys lookup. Strips the Bearer
prefix, validates the token via services/api_keys.lookup_key (which
already filters revoked keys and updates last_used_at), and returns
the user_id for the in-flight MCP request.
Tests follow the existing mock-async_session pattern in
test_api_keys.py rather than introducing a real DB fixture.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The voice_library regex's purpose is to prevent path traversal and
filter structurally-malformed IDs, not to enforce the HF catalog's
lowercase-language convention. Asserting that EN_US-amy-medium is
rejected was a category error — uppercase variants pass the regex
but would 404 at install time against HF, which is a harmless dead
end, not a security gap. Comment in the test now explains the scope.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Building on the kokoro→piper swap (B1), this adds the admin-side
voice management story so additional voices can be installed without
rebuilding the image. The bundled two voices stay as immediate defaults;
everything else is opt-in via a one-click install from the catalog.
Backend (services/voice_library.py):
- fetch_catalog() pulls voices.json from the piper-voices HF repo with
a 24h in-memory TTL. Manual refresh available via ?refresh=1 on the
library endpoint.
- shape_catalog_for_ui() projects the raw HF dict (~250 voices, lots of
nesting) into UI-friendly cards: id, name, language, country, quality,
size, install state. Sorted by language_code then name for stable
display. Install state distinguishes bundled (read-only) from user
(admin-installed, can be removed).
- install_voice() downloads .onnx + .onnx.json into /data/voices with
atomic .tmp → rename so a failed partial download can't leave a
corrupt model around. Idempotent — re-installing an already-present
voice is a no-op.
- uninstall_voice() removes /data voices; bundled /opt voices raise
PermissionError (403 at the route layer).
- Strict voice-id regex prevents path traversal in install/uninstall.
Routes (admin-only, since these write to shared /data and affect all
users on the instance):
- GET /api/voice/voices/library
- POST /api/voice/voices/install
- DELETE /api/voice/voices/<voice_id>
Frontend:
- New "Voice Library" section in Settings → Voice, visible only to
admin users. Collapsed by default; expand to load the catalog
on-demand (doesn't hammer HF for non-admins).
- Free-text filter across id, language code, language name, country,
and dataset name. Refresh button forces a catalog re-fetch.
- Per-voice row shows id, language/country/quality/speaker count, size,
and either an Install button, a Remove button (user voices), or a
"bundled" badge (read-only voices in /opt/piper-voices).
- Installs and uninstalls refresh both the library list AND the active
voice picker so the new voice is immediately selectable.
- VoiceLibraryEntry exported from api/client.ts; new client helpers
getVoiceLibrary/installVoice/uninstallVoice.
Tests:
- Pure-transformation unit tests for shape_catalog_for_ui,
_resolve_file_urls, and the voice-id regex (path-traversal coverage).
- DB/network paths (fetch_catalog, install_voice) need a real
environment — left to CI integration tests or device verification.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds an empirical surface for evaluating model swaps. One row per
assistant turn captures: model, think_enabled, tools_available,
tools_attempted, tools_succeeded, tools_failed (with error details
as JSONB). Without this, judging whether a new model "actually fires
record_moment when it should" relies on anecdote across user-reported
sessions. With it, the data is queryable directly.
Pieces:
- Migration 0046: generation_tool_log table with user_created and
per-conversation indexes.
- Model: SQLAlchemy GenerationToolLog with to_dict() for plain-dict
consumption outside session scope.
- Service: log_tool_outcomes() normalizes the in-app tool-call shape
(function/result/status) into the split buckets and persists. It
catches its own exceptions — telemetry failure must NEVER affect
the user-facing generation flow. recent_logs() helper for read.
- Integration in run_generation: called once per turn right after
log_generation, fire-and-forget.
- Tests: pure-normalization unit tests using a stub session — no DB
needed in CI. Cover the success/error split, the empty-tool-calls
case, the exception-swallowing contract, and the success=False
edge case where status incorrectly says "success".
No UI for the telemetry yet — internal infrastructure (the operator
is the consumer, not the journal user), which the FabledRulebook
"no UI no ship" explicitly excepts. Query via psql or extend the
Fable MCP later if direct shell access gets tiresome.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Prep prose (services/journal_prep.py):
- Emit explicit "WEATHER: none available — do NOT mention weather"
absent-marker so a small model can't invent partly-cloudy/temperature
prose when both configured locations have empty addresses.
- Replace negative-only system rule with positive-anchored guidance
forbidding weather/temp/precip mentions unless a numeric WEATHER
section is present; also bans echoing parenthetical labels verbatim.
- Reword overdue header to "(past their due date, still open — backlog,
not today's work)" and render lines as "was due <date>, N day(s)
overdue" with correct singular/plural. Supersedes the wording noted
in Fable task #159.
- Deterministic fabricated-weather reconciler: low-false-positive regex
detects fabricated weather phrasing; on trip with an empty section,
regenerate once with a corrective. Persistent fabrication logs ERROR
rather than mangling prose.
Journal route (routes/journal.py):
- Override message_count with len(messages) in _day_payload. The chat
path already does this; the journal path was hitting the
Conversation.to_dict() fallback to 0 because messages aren't
eager-loaded on that instance.
Tests:
- tests/test_journal_message_count.py — pins the model-level trap and
the override contract (3 cases).
- tests/test_journal_prep_hardening.py — 11 cases covering the
fabricated-weather reconciler and absent-marker rendering.
- tests/test_journal_prep_filtering.py — updated one stale assertion.
Tracks Fable task #171.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
_promote_stable_versions_for_note is the pure-function core: walks
versions chronologically and pins any with a >= AUTO_PIN_STABILITY_DAYS
(2-day) gap to the next version (or to now, for the latest). Auto-
generated label describes the stability window.
_scan_one_note loads versions for one note, runs the promotion, commits
mutations to the attached rows, then calls prune_auto_pins to cap the
auto bucket. scan_user_for_auto_pins fans out across the user's notes;
scan_all_users_for_auto_pins is the top-level entrypoint for the cron.
Per-note and per-user errors are caught and logged.
Auto-pinned versions live in their own bucket with MAX_AUTO_PINS=25 cap.
The scan job calls this after each note's promotions complete; the
oldest auto-pinned rows are dropped past the cap. Manual pins and
rolling rows are untouched.
pin_version sets pin_kind='manual' and pin_label on the target row.
Accepts already-pinned rows (promotes auto→manual, updates label).
Labels are capped at PIN_LABEL_MAX_LEN=500 chars; longer values raise
ValueError before any DB access.
unpin_version clears both fields, downgrading the row to rolling. Does
NOT delete — if the row is past the rolling FIFO depth, the next
autosave's prune will drop it.
The DELETE inside create_version now filters pin_kind IS NULL so pinned
rows (auto or manual) aren't counted toward MAX_VERSIONS=50 and aren't
candidates for deletion. Pinned versions live indefinitely regardless
of how heavy rolling autosave traffic gets on the same note.
The knowledge-note return path in create_note_tool reads note.project_id;
the SimpleNamespace fake didn't define it, so the tool crashed with
AttributeError instead of returning. The task-branch test already
included project_id; mirror that here.
create_note tool:
- New 'description' parameter accepted and forwarded to the service.
- When status is set (creating a task), 'body' is dropped before the
service call. Task bodies are owned by the consolidation pipeline.
update_note tool:
- New 'description' parameter; routed through update_fields.
- When the resolved target has is_task=True and 'body' is in the
arguments, the call errors with a message nudging toward log_work or
description. Knowledge notes are unaffected.
HTTP routes (POST/PATCH/PUT /api/notes) accept body freely — the
restriction is only at the LLM tool layer.
log_work tool now invokes maybe_consolidate(reason='log_added') after a
successful create_log. The gate inside the consolidation service handles
threshold + setting checks.
update_note service snapshots old_status before mutation and fires
maybe_consolidate(reason='task_closed') when the status transitions into
'done' or 'cancelled'. Re-saving an already-terminal status doesn't
retrigger — only transitions count.
consolidate_task reads the task title, description (read-only context),
and chronological work logs; builds a prompt via _build_consolidation_prompt;
calls generate_completion with the user's background_model setting; on a
non-empty result, writes back to Note.body, stamps consolidated_at, and
re-runs the embedding pipeline.
Errors are caught and logged. LLM failures leave body untouched so the
next trigger retries cleanly. Per-task asyncio lock prevents simultaneous
passes for the same task.
New services/consolidation.py module with maybe_consolidate() — the
debounced trigger gate. Two reasons:
- log_added: gated by DEFAULT_LOG_THRESHOLD (3) counted since the task's
consolidated_at timestamp.
- task_closed: bypasses the count gate; fires whenever status flips to
done/cancelled.
Both reasons gated by the auto_consolidate_tasks user setting (default
on). Per-task asyncio.Lock prevents two simultaneous passes for the same
task. consolidate_task is a stub here — full implementation in the next
commit.
create_note service accepts a new description kwarg and forwards it to the
Note constructor. PUT/PATCH/POST routes include description in the field
whitelist. update_note already passed **fields through setattr, so the new
column is reachable without touching that signature.
CI surfaced three issues:
- 'famous supply project' didn't substring-match 'Famous-Supply Work topics'
because the trailing filler word 'project' blocked the substring tier.
Strip {project, projects} from the query before the substring check.
- SequenceMatcher fallback against `combined` (title + description +
summary) diluted ratios to ~0.5 for plausible matches. Use title
directly; the 0.70 tier already handles description/summary mentions.
- Test patches used patch.object on a consumer module where
list_projects is imported locally — patch the source module instead.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
bvandeusen