refactor: Phase 8 — backend deletion (chat / voice / push / journal / curator)

Mega-commit. Strips all server-side LLM machinery now that Phase 7 has
removed the corresponding UI surfaces and the MCP HTTP endpoint is the
sole assistant interface.

Deleted (services/):
  chat, generation_buffer, generation_log, generation_task, llm, tools/
  (entire package), stt, tts, voice_config, voice_library, push,
  journal_closeout, journal_pipeline, journal_prep, journal_scheduler,
  journal_search, curator, curator_scheduler, consolidation,
  tag_suggestions, research, weather, article_fetcher, pending_actions,
  moments, assist, wikipedia.

Deleted (routes/):
  chat, voice, push, journal, quick_capture, fable_mcp_dist.

Deleted (models/):
  conversation, generation_tool_log, push_subscription,
  pending_curator_action, moment, weather_cache.

Deleted (tests/):
  test_generation_log, test_journal_*, test_consolidation, test_lookup_tool,
  test_notes_consolidation_trigger, test_record_moment_guards,
  test_research_pipeline, test_tools_*, test_tool_use_fixes,
  test_voice_library, test_weather_service, test_calendar_tool_tz,
  test_wikipedia.

Deleted (top-level):
  fable-mcp/ (legacy standalone stdio package — wheel-build pipeline
  also removed from Dockerfile).

app.py:
  - blueprint registrations for the 6 deleted routes
  - startup hook trimmed: no more Ollama warmup, KV-cache priming,
    journal/curator schedulers, voice model loading
  - shutdown hook simplified
  - httpx import dropped (was for Ollama calls)

pyproject.toml:
  - removed deps: pywebpush, feedparser, html2text, trafilatura
  - removed [voice] extras entirely
  - description updated for the MCP-first architecture

Dockerfile:
  - removed faster-whisper / piper-tts install steps
  - removed bundled piper voice download stage
  - removed fable-mcp wheel build stage

Surviving-file edits:
  - services/auth.py: drop Conversation table claim on first-user setup
  - services/backup.py: drop conversation / push-subscription export+restore;
    v1/v2 restore now silently skip pre-pivot conversation data
  - services/notes.py: drop maybe_consolidate trigger on task done/cancelled;
    drop _maybe_trigger_project_summary (LLM auto-summary)
  - services/projects.py: drop generate_project_summary + backfill_project_summaries
    (both LLM-driven)
  - services/user_profile.py: drop append_observations / consolidate /
    clear_learned_data (curator-tied) and build_profile_context
    (was LLM system-prompt builder)
  - services/notifications.py: stub out _fire_push_notif (was send_push_notification)
  - services/event_scheduler.py: drop event-reminder push + chat-retention
    cleanup job; keep CalDAV pull-sync + reminders job (in-app)
  - services/diagnostics.py: _curator_busy() always False
  - routes/notes.py: drop /assist, /assist/stream, /suggest-tags endpoints
  - routes/tasks.py: drop /<id>/consolidate endpoint
  - routes/settings.py: drop /models, KV-cache-prime-on-save, journal-schedule
    timezone hook, and the SearXNG search-test endpoint; inline _is_private_url
    (was in services/llm.py)
  - routes/admin.py: drop /voice, /voice/reload endpoints
  - routes/profile.py: drop /consolidate, /observations (GET, DELETE)
  - models/__init__.py: drop the 6 dead model imports

Frontend cascade:
  - stores/push.ts: deleted entirely (no callers after Phase 7)
  - stores/settings.ts: drop checkVoiceStatus + voice-status state
  - views/SettingsView.vue: drop Locations section + journalConfig state
    (was tied to /api/journal/config); drop JournalConfig + journal/voice
    api/client imports
  - frontend/api/client.ts: orphaned voice/journal/profile-observation/
    fable-mcp-dist exports are left as dead but harmless (call them and
    they 404; type-check is clean).

Pre-existing v1 backups that contained conversations/messages still
restore — those tables are silently dropped from the import path.
Anyone pulling the new image with a populated database will need the
Phase 9 migration to drop the dead tables (coming next).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-05-27 17:47:18 -04:00
parent 8bec68abc0
commit 91bafb641f
123 changed files with 161 additions and 19312 deletions
+32 -78
View File
@@ -1,49 +1,46 @@
import asyncio
"""User settings + integrations (CalDAV, SearXNG status).
Chat-model picker endpoints (/models), KV-cache priming, and journal-schedule
hooks were removed in Phase 8 alongside the chat/journal subsystems.
"""
import ipaddress
import logging
import socket
from urllib.parse import urlparse
from quart import Blueprint, jsonify, request
from fabledassistant.auth import login_required, get_current_user_id
from fabledassistant.config import Config
from fabledassistant.services.caldav import CALDAV_SETTING_KEYS, get_caldav_config, test_connection
from fabledassistant.services.llm import get_installed_models, _is_private_url
from fabledassistant.services.settings import delete_setting, get_all_settings, get_setting, set_settings_batch
logger = logging.getLogger(__name__)
async def _prime_kv_cache_bg(user_id: int, model: str) -> None:
"""Fire-and-forget: prime Ollama's KV cache with the user's system prompt."""
import httpx
from fabledassistant.services.llm import build_context, pick_num_ctx
from fabledassistant.services.tools import get_tools_for_user
def _is_private_url(url: str) -> bool:
"""SSRF-blocking helper: returns True for URLs that resolve to private,
loopback, or link-local addresses. Inlined here after services/llm.py
(the original home) was removed in Phase 8."""
try:
messages, _ = await build_context(
user_id=user_id,
history=[],
current_note_id=None,
user_message=" ",
)
# Size the prime to match what real chat requests will use, including
# tool schemas — otherwise Ollama reloads the model on the first real
# request and throws away the cache we just built.
tools = await get_tools_for_user(user_id)
num_ctx = pick_num_ctx(messages, tools=tools)
from fabledassistant.services.llm import keep_alive_for
async with httpx.AsyncClient(timeout=120.0) as client:
await client.post(
f"{Config.OLLAMA_URL}/api/chat",
json={
"model": model,
"messages": messages,
"stream": False,
"options": {"num_predict": 1, "num_ctx": num_ctx},
"keep_alive": keep_alive_for(model),
},
)
logger.info("Primed KV cache for user %d with model '%s' (num_ctx=%d)", user_id, model, num_ctx)
host = urlparse(url).hostname
if not host:
return True
# Resolve to all addresses; reject if any is private/loopback/link-local.
infos = socket.getaddrinfo(host, None)
for family, *_rest, sockaddr in infos:
ip_str = sockaddr[0]
try:
ip = ipaddress.ip_address(ip_str)
except ValueError:
continue
if ip.is_private or ip.is_loopback or ip.is_link_local or ip.is_reserved:
return True
except Exception:
logger.warning("Failed to prime KV cache for user %d", user_id, exc_info=True)
# Conservative: if we can't resolve, treat as private (reject).
return True
return False
settings_bp = Blueprint("settings", __name__, url_prefix="/api/settings")
@@ -64,29 +61,10 @@ async def update_settings_route():
if not isinstance(data, dict):
return jsonify({"error": "Expected a JSON object"}), 400
# Normalize model names to lowercase before validation. Ollama's /api/tags
# preserves whatever casing was used at pull time, but /api/chat rejects
# mixed-case tags — lowercasing here keeps the two paths consistent and
# means the stored setting is always in a form Ollama will actually accept.
_MODEL_KEYS = frozenset({"default_model", "background_model"})
for _key in _MODEL_KEYS:
if _key in data and data[_key]:
data[_key] = str(data[_key]).lower()
if "default_model" in data:
installed = await get_installed_models()
if data["default_model"]:
model = str(data["default_model"])
if installed and model not in installed:
return jsonify({"error": f"Model '{model}' is not installed"}), 400
# Empty string for model keys means "reset to system default".
# Delete the DB row so get_setting() falls back to Config defaults
# rather than returning "" and breaking model resolution everywhere.
to_save = {}
for k, v in data.items():
str_v = str(v)
if k in _MODEL_KEYS and not str_v:
if not str_v:
await delete_setting(uid, k)
else:
to_save[k] = str_v
@@ -94,29 +72,10 @@ async def update_settings_route():
if to_save:
await set_settings_batch(uid, to_save)
# Live-reschedule the journal daily-prep job when the timezone changes.
if "user_timezone" in to_save:
from fabledassistant.services.journal_scheduler import update_user_schedule as _update_journal_schedule
await _update_journal_schedule(uid)
if "default_model" in to_save and to_save["default_model"]:
asyncio.create_task(_prime_kv_cache_bg(uid, to_save["default_model"]))
settings = await get_all_settings(uid)
return jsonify(settings)
@settings_bp.route("/models", methods=["GET"])
@login_required
async def get_models_route():
"""Return installed Ollama models and the configured defaults."""
models = sorted(await get_installed_models())
return jsonify({
"models": models,
"default_chat_model": Config.OLLAMA_MODEL,
})
@settings_bp.route("/caldav", methods=["GET"])
@login_required
async def get_caldav():
@@ -166,12 +125,7 @@ async def test_caldav():
@settings_bp.route("/search", methods=["GET"])
@login_required
async def test_search():
"""Test SearXNG connectivity and preview results for a query."""
"""Report SearXNG configuration status (used by the Integrations tab)."""
if not Config.searxng_enabled():
return jsonify({"configured": False, "results": [], "searxng_url": ""})
q = request.args.get("q", "").strip()
if not q:
return jsonify({"configured": True, "results": [], "searxng_url": Config.SEARXNG_URL})
from fabledassistant.services.research import _search_searxng
results = await _search_searxng(q)
return jsonify({"configured": True, "results": results, "query": q, "searxng_url": Config.SEARXNG_URL})
return jsonify({"configured": True, "results": [], "searxng_url": Config.SEARXNG_URL})