v26.05.30.1 — MCP-first pivot + Rulebook + Plans + Soft-delete #51

2026-05-29T22:50:24-04:00

bvandeusen commented

2026-05-29 22:50:24 -04:00

(Migrated from git.fabledsword.com)

118 commits since v26.05.13.1. Headline: the MCP-first pivot lands in production, and three new feature arcs land on top of it.

MCP-first pivot (Phases 7-10)

Scribe is now a pure data store. Claude via the in-app /mcp endpoint is the only LLM. ~30,000 lines deleted across backend, frontend, and infrastructure.

New endpoint — /mcp mounted alongside /api/* via FastMCP's streamable-HTTP transport. ASGI middleware authenticates via bearer tokens from the existing api_keys table; rejects with 401 before FastMCP sees the request. Per-request user_id ContextVar for tool handlers.
34 MCP tools covering notes, tasks, projects, milestones, events, typed entities (person/place/list), search.
Embeddings: Ollama → fastembed. In-process ONNX (BAAI/bge-small-en-v1.5, 384d). First boot downloads the model to /data/fastembed-cache once. Migration 0052 cleared the existing 768d note_embeddings; startup re-embeds existing notes.
Phase 7 — frontend strip: removed /chat, /journal, /workspace/:projectId, voice overlay, the chat-style home view. Surfaces that survived: /knowledge, /notes, /tasks, /projects, /calendar, /graph, /settings, /rules, /trash.
Phase 8 — backend deletion: /api/chat/*, /api/journal/*, /api/voice/*, /api/push/*, /api/quick-capture, /api/fable-mcp/* and their backing services. The standalone fable-mcp stdio wheel removed.
Phase 9 (migration 0053): dropped conversations, messages, generation_tool_log, moments + junction tables, moment_embeddings, pending_curator_actions, push_subscriptions, weather_cache, rss_item_embeddings.
Phase 10 (migration 0054): dropped image_cache; removed Ollama service config and frontend orphans.
Rebrand: "Scribe" user-facing; fabledassistant internal package name kept stable.
Settings → MCP Access tab: create write or read-only tokens (fmcp_<32-byte-base64url> format), shown once, sent as Authorization: Bearer <token>. Configurable MCP server name + scope.

Rulebook system (May 2026 milestone)

Cross-project standards: Rulebook → Topic → Rule, sibling to Project, with project subscriptions.

Migration 0055 — rulebooks, rulebook_topics, rules, project_rulebook_subscriptions. Each rule is a statement + why + how_to_apply triple.
16 MCP tools for rulebook/topic/rule CRUD plus subscribe/unsubscribe. get_project(id) returns applicable_rules + subscribed_rulebooks (capped at 50).
/rules three-pane UI (rulebooks → topics → rules) with rule editor; project Rules tab grouped by rulebook → topic.
Seeded: the external FabledRulebook (12 topic .md files) imported into the "FabledSword family" rulebook — 12 topics, 43 rules. Fabled Assistant project subscribed.

Plans-as-tasks (May 2026 milestone)

Plans are first-class without a new entity: a Task with task_kind="plan".

Migration 0056 — additive notes.task_kind column (work | plan, default work, CHECK-constrained).
MCP: start_planning(project_id, title) seeds a plan template + returns applicable_rules. get_task surfaces applicable_rules for plan-tasks only.
UI: "Start planning" button on /projects/:id; Applicable Rules panel in the task editor; Knowledge view's Plans facet + plan badge.
Rulebook hook: rule 27 reframed ("Planning artifacts live in Scribe, never in git"); ritual rule 45 added ("Build plans via start_planning, not .md") — the rulebook itself points planning at Scribe.

Soft-delete / Trash / Retention (May 2026 milestone)

Deletes become recoverable.

Migration 0057 — deleted_at TIMESTAMPTZ NULL + deleted_batch_id TEXT NULL (+ ix_<t>_deleted_at) on 7 tables (notes, events, projects, milestones, rulebooks, rulebook_topics, rules). SoftDeleteMixin in models/base.py.
Cascade: project → milestones + tasks/notes; rulebook → topics → rules; restore brings the whole batch back together (reverses the prior FK SET-NULL behavior so a restored project comes back with its children).
MCP surface: existing delete_* flipped to soft (return {deleted_batch_id, message}); added delete_task, delete_project, delete_milestone (closes the CRUD-asymmetry gap); added list_trash, restore, purge_trash (confirmed-gated hard delete).
REST: /api/trash (GET list, POST /<batch>/restore, DELETE /<batch> for purge). All entity DELETE handlers flipped to call trash.delete.
Retention: trash_retention_days setting (default 90, 0 = keep forever). Daily 03:30 UTC APScheduler purge cron.
UI: /trash view (batches summary, Restore, Delete-permanently, Empty trash, empty state) + Settings → General → "Trash retention" field. g+x keyboard shortcut.

MCP transport — stateless HTTP

mcp/server.py sets stateless_http=True on the FastMCP instance. The default stateful streamable-HTTP transport handed Claude Code an Mcp-Session-Id; after a container redeploy the server rejected the now-unknown id with 404, and Claude Code doesn't re-initialize on 404 (issue #60949), so the connection stranded until manual /mcp retry. Stateless makes every request self-contained (bearer-auth only); the next message after a redeploy reconnects automatically — no manual intervention.

Pre-pivot work in the diff (removed by the pivot)

The diff includes substantial work that landed on dev before the MCP pivot and was subsequently removed by it: curator phases C1-C5 (additive-only curator tools, authority routing, Needs Review panel), journal closeout + captures panel, TTS migration (kokoro → piper) + voice admin UI, diagnostics for crash investigation, model-bench tooling (bench_ollama.py), user-controlled think mode. These commits remain in history but the surfaces they touched no longer exist in production.

Deploy notes

Destructive migrations. 0052/0053/0054 drop large chunks of pre-pivot data (chat conversations, journal moments, voice settings, weather cache, image cache, push subscriptions, rss item embeddings). Back up before deploy if anything in those tables matters.
Migration sequence: 0044 → 0057 run automatically on first boot.
Compose stack reduces to two services: app + db. No Ollama, no Redis, no background worker.
MCP access: create a bearer token via Settings → MCP Access after deploy. Configure in Claude Code via claude mcp add --transport http scribe <BASE_URL>/mcp with the token in the bearer header.
First-boot fastembed download: ~150 MB ONNX model goes to /data/fastembed-cache; persists across container restarts.

Verification

All backend pytest tests pass (CI green on commit 2f577fe).
/trash UI + Settings retention field device-checked by operator 2026-05-29.
MCP reconnect verified after the stateless-HTTP fix (Portainer redeploy → next message reconnected without manual /mcp retry).
Trash dogfood (via MCP this session): throwaway project cascade-delete → list_trash → restore → re-delete → purge_trash round-trip; single delete_task → list_trash → restore round-trip. Both passed.

Known follow-ups

Captured as plan-task 183 on project 3, milestone M1 "Authorization & share-correctness (May 2026)":

Share-debt in soft-delete: list_trash / restore / purge ignore user_id (cross-user leak under multi-user); retention reads user 1's setting globally rather than a per-instance admin value; collaborator-delete semantics undefined.
Cosmetic papercuts: purge_trash rowcount is off-by-one when the batch holds parent + children whose DB-level FK has ON DELETE CASCADE (end state correct, count off by one); trash items show type: "note" for is_task=True tasks.
MCP write-scope enforcement — mutating tools currently ignore the read/write token scope.

These feed the M1 foundation work. See project 3 milestones for the full M1→M4 roadmap (authz → audit → review surface → retrieval quality).

118 commits since v26.05.13.1. Headline: the MCP-first pivot lands in production, and three new feature arcs land on top of it. ## MCP-first pivot (Phases 7-10) Scribe is now a **pure data store**. Claude via the in-app `/mcp` endpoint is the only LLM. ~30,000 lines deleted across backend, frontend, and infrastructure. - **New endpoint** — `/mcp` mounted alongside `/api/*` via FastMCP's streamable-HTTP transport. ASGI middleware authenticates via bearer tokens from the existing `api_keys` table; rejects with 401 before FastMCP sees the request. Per-request `user_id` ContextVar for tool handlers. - **34 MCP tools** covering notes, tasks, projects, milestones, events, typed entities (person/place/list), search. - **Embeddings: Ollama → fastembed.** In-process ONNX (`BAAI/bge-small-en-v1.5`, 384d). First boot downloads the model to `/data/fastembed-cache` once. Migration 0052 cleared the existing 768d note_embeddings; startup re-embeds existing notes. - **Phase 7 — frontend strip:** removed `/chat`, `/journal`, `/workspace/:projectId`, voice overlay, the chat-style home view. Surfaces that survived: `/knowledge`, `/notes`, `/tasks`, `/projects`, `/calendar`, `/graph`, `/settings`, `/rules`, `/trash`. - **Phase 8 — backend deletion:** `/api/chat/*`, `/api/journal/*`, `/api/voice/*`, `/api/push/*`, `/api/quick-capture`, `/api/fable-mcp/*` and their backing services. The standalone `fable-mcp` stdio wheel removed. - **Phase 9 (migration 0053):** dropped `conversations`, `messages`, `generation_tool_log`, `moments` + junction tables, `moment_embeddings`, `pending_curator_actions`, `push_subscriptions`, `weather_cache`, `rss_item_embeddings`. - **Phase 10 (migration 0054):** dropped `image_cache`; removed Ollama service config and frontend orphans. - **Rebrand:** "Scribe" user-facing; `fabledassistant` internal package name kept stable. - **Settings → MCP Access tab:** create write or read-only tokens (`fmcp_<32-byte-base64url>` format), shown once, sent as `Authorization: Bearer <token>`. Configurable MCP server name + scope. ## Rulebook system (May 2026 milestone) Cross-project standards: Rulebook → Topic → Rule, sibling to Project, with project subscriptions. - **Migration 0055** — `rulebooks`, `rulebook_topics`, `rules`, `project_rulebook_subscriptions`. Each rule is a `statement + why + how_to_apply` triple. - **16 MCP tools** for rulebook/topic/rule CRUD plus subscribe/unsubscribe. `get_project(id)` returns `applicable_rules` + `subscribed_rulebooks` (capped at 50). - **`/rules` three-pane UI** (rulebooks → topics → rules) with rule editor; project Rules tab grouped by rulebook → topic. - **Seeded:** the external FabledRulebook (12 topic .md files) imported into the "FabledSword family" rulebook — 12 topics, 43 rules. Fabled Assistant project subscribed. ## Plans-as-tasks (May 2026 milestone) Plans are first-class without a new entity: a Task with `task_kind="plan"`. - **Migration 0056** — additive `notes.task_kind` column (`work` | `plan`, default `work`, CHECK-constrained). - **MCP:** `start_planning(project_id, title)` seeds a plan template + returns applicable_rules. `get_task` surfaces `applicable_rules` for plan-tasks only. - **UI:** "Start planning" button on `/projects/:id`; Applicable Rules panel in the task editor; Knowledge view's Plans facet + plan badge. - **Rulebook hook:** rule 27 reframed ("Planning artifacts live in Scribe, never in git"); ritual rule 45 added ("Build plans via start_planning, not .md") — the rulebook itself points planning at Scribe. ## Soft-delete / Trash / Retention (May 2026 milestone) Deletes become recoverable. - **Migration 0057** — `deleted_at TIMESTAMPTZ NULL` + `deleted_batch_id TEXT NULL` (+ `ix_<t>_deleted_at`) on 7 tables (`notes`, `events`, `projects`, `milestones`, `rulebooks`, `rulebook_topics`, `rules`). `SoftDeleteMixin` in `models/base.py`. - **Cascade:** project → milestones + tasks/notes; rulebook → topics → rules; restore brings the whole batch back together (reverses the prior FK SET-NULL behavior so a restored project comes back with its children). - **MCP surface:** existing `delete_*` flipped to soft (return `{deleted_batch_id, message}`); added `delete_task`, `delete_project`, `delete_milestone` (closes the CRUD-asymmetry gap); added `list_trash`, `restore`, `purge_trash` (confirmed-gated hard delete). - **REST:** `/api/trash` (GET list, POST `/<batch>/restore`, DELETE `/<batch>` for purge). All entity DELETE handlers flipped to call `trash.delete`. - **Retention:** `trash_retention_days` setting (default 90, `0` = keep forever). Daily 03:30 UTC APScheduler purge cron. - **UI:** `/trash` view (batches summary, Restore, Delete-permanently, Empty trash, empty state) + Settings → General → "Trash retention" field. `g`+`x` keyboard shortcut. ## MCP transport — stateless HTTP `mcp/server.py` sets `stateless_http=True` on the FastMCP instance. The default stateful streamable-HTTP transport handed Claude Code an `Mcp-Session-Id`; after a container redeploy the server rejected the now-unknown id with 404, and Claude Code doesn't re-initialize on 404 (issue #60949), so the connection stranded until manual `/mcp` retry. Stateless makes every request self-contained (bearer-auth only); the next message after a redeploy reconnects automatically — no manual intervention. ## Pre-pivot work in the diff (removed by the pivot) The diff includes substantial work that landed on dev before the MCP pivot and was subsequently removed by it: curator phases C1-C5 (additive-only curator tools, authority routing, Needs Review panel), journal closeout + captures panel, TTS migration (kokoro → piper) + voice admin UI, diagnostics for crash investigation, model-bench tooling (`bench_ollama.py`), user-controlled think mode. These commits remain in history but the surfaces they touched no longer exist in production. ## Deploy notes - **Destructive migrations.** 0052/0053/0054 drop large chunks of pre-pivot data (chat conversations, journal moments, voice settings, weather cache, image cache, push subscriptions, rss item embeddings). **Back up before deploy if anything in those tables matters.** - **Migration sequence:** 0044 → 0057 run automatically on first boot. - **Compose stack reduces to two services:** `app` + `db`. No Ollama, no Redis, no background worker. - **MCP access:** create a bearer token via Settings → MCP Access after deploy. Configure in Claude Code via `claude mcp add --transport http scribe <BASE_URL>/mcp` with the token in the bearer header. - **First-boot fastembed download:** ~150 MB ONNX model goes to `/data/fastembed-cache`; persists across container restarts. ## Verification - All backend pytest tests pass (CI green on commit `2f577fe`). - `/trash` UI + Settings retention field device-checked by operator 2026-05-29. - MCP reconnect verified after the stateless-HTTP fix (Portainer redeploy → next message reconnected without manual `/mcp` retry). - Trash dogfood (via MCP this session): throwaway project cascade-delete → `list_trash` → `restore` → re-delete → `purge_trash` round-trip; single `delete_task` → `list_trash` → `restore` round-trip. Both passed. ## Known follow-ups Captured as plan-task **183** on project 3, milestone M1 "Authorization & share-correctness (May 2026)": - **Share-debt in soft-delete:** `list_trash` / `restore` / `purge` ignore `user_id` (cross-user leak under multi-user); retention reads user 1's setting globally rather than a per-instance admin value; collaborator-delete semantics undefined. - **Cosmetic papercuts:** `purge_trash` rowcount is off-by-one when the batch holds parent + children whose DB-level FK has `ON DELETE CASCADE` (end state correct, count off by one); trash items show `type: "note"` for `is_task=True` tasks. - **MCP write-scope enforcement** — mutating tools currently ignore the read/write token scope. These feed the M1 foundation work. See project 3 milestones for the full M1→M4 roadmap (authz → audit → review surface → retrieval quality).

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: bvandeusen/FabledScribe#51