refactor: Phase 8 — backend deletion (chat / voice / push / journal / curator)

Mega-commit. Strips all server-side LLM machinery now that Phase 7 has
removed the corresponding UI surfaces and the MCP HTTP endpoint is the
sole assistant interface.

Deleted (services/):
  chat, generation_buffer, generation_log, generation_task, llm, tools/
  (entire package), stt, tts, voice_config, voice_library, push,
  journal_closeout, journal_pipeline, journal_prep, journal_scheduler,
  journal_search, curator, curator_scheduler, consolidation,
  tag_suggestions, research, weather, article_fetcher, pending_actions,
  moments, assist, wikipedia.

Deleted (routes/):
  chat, voice, push, journal, quick_capture, fable_mcp_dist.

Deleted (models/):
  conversation, generation_tool_log, push_subscription,
  pending_curator_action, moment, weather_cache.

Deleted (tests/):
  test_generation_log, test_journal_*, test_consolidation, test_lookup_tool,
  test_notes_consolidation_trigger, test_record_moment_guards,
  test_research_pipeline, test_tools_*, test_tool_use_fixes,
  test_voice_library, test_weather_service, test_calendar_tool_tz,
  test_wikipedia.

Deleted (top-level):
  fable-mcp/ (legacy standalone stdio package — wheel-build pipeline
  also removed from Dockerfile).

app.py:
  - blueprint registrations for the 6 deleted routes
  - startup hook trimmed: no more Ollama warmup, KV-cache priming,
    journal/curator schedulers, voice model loading
  - shutdown hook simplified
  - httpx import dropped (was for Ollama calls)

pyproject.toml:
  - removed deps: pywebpush, feedparser, html2text, trafilatura
  - removed [voice] extras entirely
  - description updated for the MCP-first architecture

Dockerfile:
  - removed faster-whisper / piper-tts install steps
  - removed bundled piper voice download stage
  - removed fable-mcp wheel build stage

Surviving-file edits:
  - services/auth.py: drop Conversation table claim on first-user setup
  - services/backup.py: drop conversation / push-subscription export+restore;
    v1/v2 restore now silently skip pre-pivot conversation data
  - services/notes.py: drop maybe_consolidate trigger on task done/cancelled;
    drop _maybe_trigger_project_summary (LLM auto-summary)
  - services/projects.py: drop generate_project_summary + backfill_project_summaries
    (both LLM-driven)
  - services/user_profile.py: drop append_observations / consolidate /
    clear_learned_data (curator-tied) and build_profile_context
    (was LLM system-prompt builder)
  - services/notifications.py: stub out _fire_push_notif (was send_push_notification)
  - services/event_scheduler.py: drop event-reminder push + chat-retention
    cleanup job; keep CalDAV pull-sync + reminders job (in-app)
  - services/diagnostics.py: _curator_busy() always False
  - routes/notes.py: drop /assist, /assist/stream, /suggest-tags endpoints
  - routes/tasks.py: drop /<id>/consolidate endpoint
  - routes/settings.py: drop /models, KV-cache-prime-on-save, journal-schedule
    timezone hook, and the SearXNG search-test endpoint; inline _is_private_url
    (was in services/llm.py)
  - routes/admin.py: drop /voice, /voice/reload endpoints
  - routes/profile.py: drop /consolidate, /observations (GET, DELETE)
  - models/__init__.py: drop the 6 dead model imports

Frontend cascade:
  - stores/push.ts: deleted entirely (no callers after Phase 7)
  - stores/settings.ts: drop checkVoiceStatus + voice-status state
  - views/SettingsView.vue: drop Locations section + journalConfig state
    (was tied to /api/journal/config); drop JournalConfig + journal/voice
    api/client imports
  - frontend/api/client.ts: orphaned voice/journal/profile-observation/
    fable-mcp-dist exports are left as dead but harmless (call them and
    they 404; type-check is clean).

Pre-existing v1 backups that contained conversations/messages still
restore — those tables are silently dropped from the import path.
Anyone pulling the new image with a populated database will need the
Phase 9 migration to drop the dead tables (coming next).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-05-27 17:47:18 -04:00
parent 8bec68abc0
commit 91bafb641f
123 changed files with 161 additions and 19312 deletions
+7 -172
View File
@@ -1,7 +1,11 @@
"""User profile service — structured per-user preferences for LLM context."""
import asyncio
"""User profile service — structured per-user preferences.
Post-pivot the LLM-driven observation/consolidation surface is gone (curator
deleted in Phase 8). The profile model is preserved as user-managed metadata
(name, job, expertise, style, tone, interests, work schedule) — Claude can
read it via the upcoming MCP profile tools.
"""
import logging
from datetime import date, datetime, timedelta, timezone
from sqlalchemy import select
@@ -14,9 +18,6 @@ VALID_EXPERTISE = {"novice", "intermediate", "expert"}
VALID_STYLES = {"concise", "balanced", "detailed"}
VALID_TONES = {"casual", "professional", "technical"}
# Trigger consolidation when raw observations reach this count
_CONSOLIDATION_THRESHOLD = 14
async def get_profile(user_id: int) -> UserProfile:
"""Get or create the profile row for a user."""
@@ -53,169 +54,3 @@ async def update_profile(user_id: int, data: dict) -> UserProfile:
await session.commit()
await session.refresh(profile)
return profile
async def append_observations(user_id: int, bullets: str) -> None:
"""
Append a new dated observation entry from the day's briefing closeout.
Automatically triggers consolidation when the raw list grows large.
"""
if not bullets.strip():
return
async with async_session() as session:
result = await session.execute(
select(UserProfile).where(UserProfile.user_id == user_id)
)
profile = result.scalar_one_or_none()
if profile is None:
profile = UserProfile(user_id=user_id)
session.add(profile)
existing: list = list(profile.observations_raw or [])
existing.append({
"date": date.today().isoformat(),
"bullets": bullets.strip(),
})
# Keep at most 60 raw entries as a rolling window
profile.observations_raw = existing[-60:]
profile.observations_updated_at = datetime.now(timezone.utc)
await session.commit()
raw_count = len(profile.observations_raw or [])
logger.info("Appended observations for user %d (%d raw entries)", user_id, raw_count)
if raw_count >= _CONSOLIDATION_THRESHOLD:
asyncio.create_task(_consolidate_observations(user_id))
async def consolidate_observations(user_id: int) -> str:
"""Public entry point to manually trigger observation consolidation."""
return await _consolidate_observations(user_id)
async def clear_learned_data(user_id: int) -> None:
"""Reset all learned observations and summary for a user."""
async with async_session() as session:
result = await session.execute(
select(UserProfile).where(UserProfile.user_id == user_id)
)
profile = result.scalar_one_or_none()
if profile:
profile.learned_summary = None
profile.observations_raw = []
profile.observations_updated_at = None
await session.commit()
async def _consolidate_observations(user_id: int) -> str:
"""
LLM pass: synthesise all raw observation bullets into an updated
learned_summary paragraph. Prunes raw entries older than 30 days afterwards.
"""
from fabledassistant.config import Config
from fabledassistant.services.llm import generate_completion
from fabledassistant.services.settings import get_setting
async with async_session() as session:
result = await session.execute(
select(UserProfile).where(UserProfile.user_id == user_id)
)
profile = result.scalar_one_or_none()
if not profile or not profile.observations_raw:
return ""
observations = list(profile.observations_raw)
existing_summary = profile.learned_summary or ""
obs_text = "\n\n".join(
f"[{entry['date']}]\n{entry['bullets']}"
for entry in observations
)
system = (
"You are synthesising preference observations into a concise user profile summary. "
"Consolidate the observations into 3-6 factual sentences describing the user's patterns, "
"preferences, and habits. Be specific and useful for a personal assistant. "
"Merge with any existing summary, removing duplicates and outdated information. "
"Output only the consolidated summary paragraph — no preamble, no bullet points."
)
user_prompt = ""
if existing_summary:
user_prompt += f"Existing summary:\n{existing_summary}\n\n"
user_prompt += f"New observations:\n{obs_text}"
# Profile observation consolidation reasons over multiple documents to
# produce a coherent summary — closer to curator-shaped work than chat.
# Route to worker (background_model). Falls back to OLLAMA_MODEL only if
# neither setting nor BACKGROUND default is available.
model = (
await get_setting(user_id, "background_model", "")
or Config.OLLAMA_BACKGROUND_MODEL
or Config.OLLAMA_MODEL
)
try:
new_summary = (await generate_completion(
[
{"role": "system", "content": system},
{"role": "user", "content": user_prompt},
],
model,
)).strip()
except Exception:
logger.warning("Observation consolidation failed for user %d", user_id, exc_info=True)
new_summary = ""
if new_summary:
cutoff = (date.today() - timedelta(days=30)).isoformat()
async with async_session() as session:
result = await session.execute(
select(UserProfile).where(UserProfile.user_id == user_id)
)
profile = result.scalar_one_or_none()
if profile:
profile.learned_summary = new_summary
profile.observations_raw = [
o for o in (profile.observations_raw or [])
if o.get("date", "") >= cutoff
]
await session.commit()
logger.info("Consolidated observations for user %d", user_id)
return new_summary
async def build_profile_context(user_id: int) -> str:
"""
Build a formatted context string from the user's structured profile
for injection into LLM system prompts (briefing and chat).
Returns an empty string if no meaningful data is set.
"""
profile = await get_profile(user_id)
parts: list[str] = []
if profile.display_name:
parts.append(f"User's name: {profile.display_name}")
if profile.job_title or profile.industry:
job = " in ".join(filter(None, [profile.job_title, profile.industry]))
parts.append(f"Occupation: {job}")
if profile.expertise_level and profile.expertise_level != "intermediate":
parts.append(
f"Expertise level: {profile.expertise_level} — calibrate explanation depth accordingly"
)
if profile.response_style or profile.tone:
style = profile.response_style or "balanced"
tone = profile.tone or "casual"
parts.append(f"Preferred response style: {style}, tone: {tone}")
if profile.interests:
parts.append(f"Interests: {', '.join(profile.interests)}")
if profile.work_schedule:
sched = profile.work_schedule
days = ", ".join(sched.get("days") or []) or "weekdays"
start = sched.get("start", "9:00")
end = sched.get("end", "17:00")
parts.append(f"Work schedule: {days}, {start}{end}")
if profile.learned_summary:
parts.append(f"What the assistant has learned about this user: {profile.learned_summary}")
return "\n".join(parts)