bvandeusen/FabledScribe
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 50 Wiki Activity

refactor: Phase 8 — backend deletion (chat / voice / push / journal / curator)

Browse Source 
Mega-commit. Strips all server-side LLM machinery now that Phase 7 has
removed the corresponding UI surfaces and the MCP HTTP endpoint is the
sole assistant interface.

Deleted (services/):
  chat, generation_buffer, generation_log, generation_task, llm, tools/
  (entire package), stt, tts, voice_config, voice_library, push,
  journal_closeout, journal_pipeline, journal_prep, journal_scheduler,
  journal_search, curator, curator_scheduler, consolidation,
  tag_suggestions, research, weather, article_fetcher, pending_actions,
  moments, assist, wikipedia.

Deleted (routes/):
  chat, voice, push, journal, quick_capture, fable_mcp_dist.

Deleted (models/):
  conversation, generation_tool_log, push_subscription,
  pending_curator_action, moment, weather_cache.

Deleted (tests/):
  test_generation_log, test_journal_*, test_consolidation, test_lookup_tool,
  test_notes_consolidation_trigger, test_record_moment_guards,
  test_research_pipeline, test_tools_*, test_tool_use_fixes,
  test_voice_library, test_weather_service, test_calendar_tool_tz,
  test_wikipedia.

Deleted (top-level):
  fable-mcp/ (legacy standalone stdio package — wheel-build pipeline
  also removed from Dockerfile).

app.py:
  - blueprint registrations for the 6 deleted routes
  - startup hook trimmed: no more Ollama warmup, KV-cache priming,
    journal/curator schedulers, voice model loading
  - shutdown hook simplified
  - httpx import dropped (was for Ollama calls)

pyproject.toml:
  - removed deps: pywebpush, feedparser, html2text, trafilatura
  - removed [voice] extras entirely
  - description updated for the MCP-first architecture

Dockerfile:
  - removed faster-whisper / piper-tts install steps
  - removed bundled piper voice download stage
  - removed fable-mcp wheel build stage

Surviving-file edits:
  - services/auth.py: drop Conversation table claim on first-user setup
  - services/backup.py: drop conversation / push-subscription export+restore;
    v1/v2 restore now silently skip pre-pivot conversation data
  - services/notes.py: drop maybe_consolidate trigger on task done/cancelled;
    drop _maybe_trigger_project_summary (LLM auto-summary)
  - services/projects.py: drop generate_project_summary + backfill_project_summaries
    (both LLM-driven)
  - services/user_profile.py: drop append_observations / consolidate /
    clear_learned_data (curator-tied) and build_profile_context
    (was LLM system-prompt builder)
  - services/notifications.py: stub out _fire_push_notif (was send_push_notification)
  - services/event_scheduler.py: drop event-reminder push + chat-retention
    cleanup job; keep CalDAV pull-sync + reminders job (in-app)
  - services/diagnostics.py: _curator_busy() always False
  - routes/notes.py: drop /assist, /assist/stream, /suggest-tags endpoints
  - routes/tasks.py: drop /<id>/consolidate endpoint
  - routes/settings.py: drop /models, KV-cache-prime-on-save, journal-schedule
    timezone hook, and the SearXNG search-test endpoint; inline _is_private_url
    (was in services/llm.py)
  - routes/admin.py: drop /voice, /voice/reload endpoints
  - routes/profile.py: drop /consolidate, /observations (GET, DELETE)
  - models/__init__.py: drop the 6 dead model imports

Frontend cascade:
  - stores/push.ts: deleted entirely (no callers after Phase 7)
  - stores/settings.ts: drop checkVoiceStatus + voice-status state
  - views/SettingsView.vue: drop Locations section + journalConfig state
    (was tied to /api/journal/config); drop JournalConfig + journal/voice
    api/client imports
  - frontend/api/client.ts: orphaned voice/journal/profile-observation/
    fable-mcp-dist exports are left as dead but harmless (call them and
    they 404; type-check is clean).

Pre-existing v1 backups that contained conversations/messages still
restore — those tables are silently dropped from the import path.
Anyone pulling the new image with a populated database will need the
Phase 9 migration to drop the dead tables (coming next).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Bryan Van Deusen
2026-05-27 17:47:18 -04:00
parent 8bec68abc0
commit 91bafb641f
123 changed files with 161 additions and 19312 deletions
Download Patch File Download Diff File Expand all files Collapse all files
src/fabledassistant/services/backup.py
+14 -127
View File
@@ -2,16 +2,13 @@ import logging
from datetime import date, datetime, timezone
from sqlalchemy import select
from sqlalchemy.orm import selectinload
from fabledassistant.models import async_session
from fabledassistant.models.conversation import Conversation, Message
from fabledassistant.models.milestone import Milestone
from fabledassistant.models.note import Note
from fabledassistant.models.note_draft import NoteDraft
from fabledassistant.models.note_version import NoteVersion
from fabledassistant.models.project import Project
from fabledassistant.models.push_subscription import PushSubscription
from fabledassistant.models.setting import Setting
from fabledassistant.models.task_log import TaskLog
from fabledassistant.models.user import User
@@ -43,18 +40,14 @@ async def export_full_backup() -> dict:
note_versions = (await session.execute(
select(NoteVersion).order_by(NoteVersion.note_id, NoteVersion.id)
)).scalars().all()
conversations = (await session.execute(
select(Conversation).options(selectinload(Conversation.messages))
)).scalars().all()
settings = (await session.execute(select(Setting))).scalars().all()
push_subs = (await session.execute(select(PushSubscription))).scalars().all()
return {
"version": 2,
"scope": "full",
"exported_at": datetime.now(timezone.utc).isoformat(),
"_security_notice": (
"This backup contains hashed passwords and push subscription keys. "
"This backup contains hashed passwords. "
"Store it securely and restrict access."
),
"users": [
@@ -156,44 +149,10 @@ async def export_full_backup() -> dict:
}
for nv in note_versions
],
"conversations": [
{
"id": c.id,
"user_id": c.user_id,
"title": c.title,
"model": c.model,
"created_at": c.created_at.isoformat(),
"updated_at": c.updated_at.isoformat(),
"messages": [
{
"id": m.id,
"role": m.role,
"content": m.content,
"status": m.status,
"context_note_id": m.context_note_id,
"tool_calls": m.tool_calls,
"created_at": m.created_at.isoformat(),
}
for m in c.messages
],
}
for c in conversations
],
"settings": [
{"user_id": s.user_id, "key": s.key, "value": s.value}
for s in settings
],
"push_subscriptions": [
{
"user_id": ps.user_id,
"endpoint": ps.endpoint,
"p256dh": ps.p256dh,
"auth": ps.auth,
"user_agent": ps.user_agent,
"created_at": ps.created_at.isoformat(),
}
for ps in push_subs
],
}
@@ -220,11 +179,6 @@ async def export_user_backup(user_id: int) -> dict:
select(NoteVersion).where(NoteVersion.user_id == user_id)
.order_by(NoteVersion.note_id, NoteVersion.id)
)).scalars().all()
conversations = (await session.execute(
select(Conversation)
.options(selectinload(Conversation.messages))
.where(Conversation.user_id == user_id)
)).scalars().all()
settings = (await session.execute(
select(Setting).where(Setting.user_id == user_id)
)).scalars().all()
@@ -322,28 +276,6 @@ async def export_user_backup(user_id: int) -> dict:
}
for nv in note_versions
],
"conversations": [
{
"id": c.id,
"title": c.title,
"model": c.model,
"created_at": c.created_at.isoformat(),
"updated_at": c.updated_at.isoformat(),
"messages": [
{
"id": m.id,
"role": m.role,
"content": m.content,
"status": m.status,
"context_note_id": m.context_note_id,
"tool_calls": m.tool_calls,
"created_at": m.created_at.isoformat(),
}
for m in c.messages
],
}
for c in conversations
],
"settings": [
{"key": s.key, "value": s.value}
for s in settings
@@ -364,8 +296,12 @@ async def restore_full_backup(data: dict) -> dict:
async def _restore_v1(data: dict) -> dict:
"""Restore legacy v1 backup (original format)."""
stats = {"users": 0, "notes": 0, "conversations": 0, "messages": 0, "settings": 0}
"""Restore legacy v1 backup (original format).
Pre-pivot v1 backups included conversations + messages; those are
skipped during restore now that the chat subsystem is gone.
"""
stats = {"users": 0, "notes": 0, "settings": 0}
async with async_session() as session:
user_id_map: dict[int, int] = {}
@@ -416,31 +352,6 @@ async def _restore_v1(data: dict) -> dict:
if note_row:
note_row.parent_id = note_id_map[old_parent]
for c_data in data.get("conversations", []):
mapped_user_id = user_id_map.get(c_data.get("user_id", 0))
if mapped_user_id is None:
continue
conv = Conversation(
user_id=mapped_user_id,
title=c_data.get("title", ""),
model=c_data.get("model", ""),
created_at=_dt(c_data.get("created_at")),
updated_at=_dt(c_data.get("updated_at")),
)
session.add(conv)
await session.flush()
stats["conversations"] += 1
for m_data in c_data.get("messages", []):
msg = Message(
conversation_id=conv.id,
role=m_data["role"],
content=m_data.get("content", ""),
context_note_id=note_id_map.get(m_data["context_note_id"]) if m_data.get("context_note_id") else None,
created_at=_dt(m_data.get("created_at")),
)
session.add(msg)
stats["messages"] += 1
for s_data in data.get("settings", []):
mapped_user_id = user_id_map.get(s_data.get("user_id", 0))
if mapped_user_id is None:
@@ -455,11 +366,15 @@ async def _restore_v1(data: dict) -> dict:
async def _restore_v2(data: dict) -> dict:
"""Restore v2 backup with full FK re-mapping."""
"""Restore v2 backup with full FK re-mapping.
Conversations + push subscriptions in pre-pivot backups are silently
skipped — those subsystems were removed in the MCP-first pivot.
"""
stats: dict[str, int] = {
"users": 0, "projects": 0, "milestones": 0, "notes": 0,
"task_logs": 0, "note_drafts": 0, "note_versions": 0,
"conversations": 0, "messages": 0, "settings": 0,
"settings": 0,
}
async with async_session() as session:
@@ -616,35 +531,7 @@ async def _restore_v2(data: dict) -> dict:
session.add(nv)
stats["note_versions"] += 1
# 8. Conversations + Messages
for c_data in data.get("conversations", []):
mapped_uid = user_id_map.get(c_data.get("user_id", 0))
if mapped_uid is None:
continue
conv = Conversation(
user_id=mapped_uid,
title=c_data.get("title", ""),
model=c_data.get("model", ""),
created_at=_dt(c_data.get("created_at")),
updated_at=_dt(c_data.get("updated_at")),
)
session.add(conv)
await session.flush()
stats["conversations"] += 1
for m_data in c_data.get("messages", []):
msg = Message(
conversation_id=conv.id,
role=m_data["role"],
content=m_data.get("content", ""),
status=m_data.get("status", "complete"),
context_note_id=note_id_map.get(m_data["context_note_id"]) if m_data.get("context_note_id") else None,
tool_calls=m_data.get("tool_calls"),
created_at=_dt(m_data.get("created_at")),
)
session.add(msg)
stats["messages"] += 1
# 9. Settings
# 8. Settings
for s_data in data.get("settings", []):
mapped_uid = user_id_map.get(s_data.get("user_id", 0))
if mapped_uid is None: