refactor: Phase 8 — backend deletion (chat / voice / push / journal / curator)
Mega-commit. Strips all server-side LLM machinery now that Phase 7 has
removed the corresponding UI surfaces and the MCP HTTP endpoint is the
sole assistant interface.
Deleted (services/):
chat, generation_buffer, generation_log, generation_task, llm, tools/
(entire package), stt, tts, voice_config, voice_library, push,
journal_closeout, journal_pipeline, journal_prep, journal_scheduler,
journal_search, curator, curator_scheduler, consolidation,
tag_suggestions, research, weather, article_fetcher, pending_actions,
moments, assist, wikipedia.
Deleted (routes/):
chat, voice, push, journal, quick_capture, fable_mcp_dist.
Deleted (models/):
conversation, generation_tool_log, push_subscription,
pending_curator_action, moment, weather_cache.
Deleted (tests/):
test_generation_log, test_journal_*, test_consolidation, test_lookup_tool,
test_notes_consolidation_trigger, test_record_moment_guards,
test_research_pipeline, test_tools_*, test_tool_use_fixes,
test_voice_library, test_weather_service, test_calendar_tool_tz,
test_wikipedia.
Deleted (top-level):
fable-mcp/ (legacy standalone stdio package — wheel-build pipeline
also removed from Dockerfile).
app.py:
- blueprint registrations for the 6 deleted routes
- startup hook trimmed: no more Ollama warmup, KV-cache priming,
journal/curator schedulers, voice model loading
- shutdown hook simplified
- httpx import dropped (was for Ollama calls)
pyproject.toml:
- removed deps: pywebpush, feedparser, html2text, trafilatura
- removed [voice] extras entirely
- description updated for the MCP-first architecture
Dockerfile:
- removed faster-whisper / piper-tts install steps
- removed bundled piper voice download stage
- removed fable-mcp wheel build stage
Surviving-file edits:
- services/auth.py: drop Conversation table claim on first-user setup
- services/backup.py: drop conversation / push-subscription export+restore;
v1/v2 restore now silently skip pre-pivot conversation data
- services/notes.py: drop maybe_consolidate trigger on task done/cancelled;
drop _maybe_trigger_project_summary (LLM auto-summary)
- services/projects.py: drop generate_project_summary + backfill_project_summaries
(both LLM-driven)
- services/user_profile.py: drop append_observations / consolidate /
clear_learned_data (curator-tied) and build_profile_context
(was LLM system-prompt builder)
- services/notifications.py: stub out _fire_push_notif (was send_push_notification)
- services/event_scheduler.py: drop event-reminder push + chat-retention
cleanup job; keep CalDAV pull-sync + reminders job (in-app)
- services/diagnostics.py: _curator_busy() always False
- routes/notes.py: drop /assist, /assist/stream, /suggest-tags endpoints
- routes/tasks.py: drop /<id>/consolidate endpoint
- routes/settings.py: drop /models, KV-cache-prime-on-save, journal-schedule
timezone hook, and the SearXNG search-test endpoint; inline _is_private_url
(was in services/llm.py)
- routes/admin.py: drop /voice, /voice/reload endpoints
- routes/profile.py: drop /consolidate, /observations (GET, DELETE)
- models/__init__.py: drop the 6 dead model imports
Frontend cascade:
- stores/push.ts: deleted entirely (no callers after Phase 7)
- stores/settings.ts: drop checkVoiceStatus + voice-status state
- views/SettingsView.vue: drop Locations section + journalConfig state
(was tied to /api/journal/config); drop JournalConfig + journal/voice
api/client imports
- frontend/api/client.ts: orphaned voice/journal/profile-observation/
fable-mcp-dist exports are left as dead but harmless (call them and
they 404; type-check is clean).
Pre-existing v1 backups that contained conversations/messages still
restore — those tables are silently dropped from the import path.
Anyone pulling the new image with a populated database will need the
Phase 9 migration to drop the dead tables (coming next).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -1,7 +1,11 @@
|
||||
"""User profile service — structured per-user preferences for LLM context."""
|
||||
import asyncio
|
||||
"""User profile service — structured per-user preferences.
|
||||
|
||||
Post-pivot the LLM-driven observation/consolidation surface is gone (curator
|
||||
deleted in Phase 8). The profile model is preserved as user-managed metadata
|
||||
(name, job, expertise, style, tone, interests, work schedule) — Claude can
|
||||
read it via the upcoming MCP profile tools.
|
||||
"""
|
||||
import logging
|
||||
from datetime import date, datetime, timedelta, timezone
|
||||
|
||||
from sqlalchemy import select
|
||||
|
||||
@@ -14,9 +18,6 @@ VALID_EXPERTISE = {"novice", "intermediate", "expert"}
|
||||
VALID_STYLES = {"concise", "balanced", "detailed"}
|
||||
VALID_TONES = {"casual", "professional", "technical"}
|
||||
|
||||
# Trigger consolidation when raw observations reach this count
|
||||
_CONSOLIDATION_THRESHOLD = 14
|
||||
|
||||
|
||||
async def get_profile(user_id: int) -> UserProfile:
|
||||
"""Get or create the profile row for a user."""
|
||||
@@ -53,169 +54,3 @@ async def update_profile(user_id: int, data: dict) -> UserProfile:
|
||||
await session.commit()
|
||||
await session.refresh(profile)
|
||||
return profile
|
||||
|
||||
|
||||
async def append_observations(user_id: int, bullets: str) -> None:
|
||||
"""
|
||||
Append a new dated observation entry from the day's briefing closeout.
|
||||
Automatically triggers consolidation when the raw list grows large.
|
||||
"""
|
||||
if not bullets.strip():
|
||||
return
|
||||
|
||||
async with async_session() as session:
|
||||
result = await session.execute(
|
||||
select(UserProfile).where(UserProfile.user_id == user_id)
|
||||
)
|
||||
profile = result.scalar_one_or_none()
|
||||
if profile is None:
|
||||
profile = UserProfile(user_id=user_id)
|
||||
session.add(profile)
|
||||
|
||||
existing: list = list(profile.observations_raw or [])
|
||||
existing.append({
|
||||
"date": date.today().isoformat(),
|
||||
"bullets": bullets.strip(),
|
||||
})
|
||||
# Keep at most 60 raw entries as a rolling window
|
||||
profile.observations_raw = existing[-60:]
|
||||
profile.observations_updated_at = datetime.now(timezone.utc)
|
||||
await session.commit()
|
||||
raw_count = len(profile.observations_raw or [])
|
||||
|
||||
logger.info("Appended observations for user %d (%d raw entries)", user_id, raw_count)
|
||||
|
||||
if raw_count >= _CONSOLIDATION_THRESHOLD:
|
||||
asyncio.create_task(_consolidate_observations(user_id))
|
||||
|
||||
|
||||
async def consolidate_observations(user_id: int) -> str:
|
||||
"""Public entry point to manually trigger observation consolidation."""
|
||||
return await _consolidate_observations(user_id)
|
||||
|
||||
|
||||
async def clear_learned_data(user_id: int) -> None:
|
||||
"""Reset all learned observations and summary for a user."""
|
||||
async with async_session() as session:
|
||||
result = await session.execute(
|
||||
select(UserProfile).where(UserProfile.user_id == user_id)
|
||||
)
|
||||
profile = result.scalar_one_or_none()
|
||||
if profile:
|
||||
profile.learned_summary = None
|
||||
profile.observations_raw = []
|
||||
profile.observations_updated_at = None
|
||||
await session.commit()
|
||||
|
||||
|
||||
async def _consolidate_observations(user_id: int) -> str:
|
||||
"""
|
||||
LLM pass: synthesise all raw observation bullets into an updated
|
||||
learned_summary paragraph. Prunes raw entries older than 30 days afterwards.
|
||||
"""
|
||||
from fabledassistant.config import Config
|
||||
from fabledassistant.services.llm import generate_completion
|
||||
from fabledassistant.services.settings import get_setting
|
||||
|
||||
async with async_session() as session:
|
||||
result = await session.execute(
|
||||
select(UserProfile).where(UserProfile.user_id == user_id)
|
||||
)
|
||||
profile = result.scalar_one_or_none()
|
||||
if not profile or not profile.observations_raw:
|
||||
return ""
|
||||
observations = list(profile.observations_raw)
|
||||
existing_summary = profile.learned_summary or ""
|
||||
|
||||
obs_text = "\n\n".join(
|
||||
f"[{entry['date']}]\n{entry['bullets']}"
|
||||
for entry in observations
|
||||
)
|
||||
|
||||
system = (
|
||||
"You are synthesising preference observations into a concise user profile summary. "
|
||||
"Consolidate the observations into 3-6 factual sentences describing the user's patterns, "
|
||||
"preferences, and habits. Be specific and useful for a personal assistant. "
|
||||
"Merge with any existing summary, removing duplicates and outdated information. "
|
||||
"Output only the consolidated summary paragraph — no preamble, no bullet points."
|
||||
)
|
||||
user_prompt = ""
|
||||
if existing_summary:
|
||||
user_prompt += f"Existing summary:\n{existing_summary}\n\n"
|
||||
user_prompt += f"New observations:\n{obs_text}"
|
||||
|
||||
# Profile observation consolidation reasons over multiple documents to
|
||||
# produce a coherent summary — closer to curator-shaped work than chat.
|
||||
# Route to worker (background_model). Falls back to OLLAMA_MODEL only if
|
||||
# neither setting nor BACKGROUND default is available.
|
||||
model = (
|
||||
await get_setting(user_id, "background_model", "")
|
||||
or Config.OLLAMA_BACKGROUND_MODEL
|
||||
or Config.OLLAMA_MODEL
|
||||
)
|
||||
try:
|
||||
new_summary = (await generate_completion(
|
||||
[
|
||||
{"role": "system", "content": system},
|
||||
{"role": "user", "content": user_prompt},
|
||||
],
|
||||
model,
|
||||
)).strip()
|
||||
except Exception:
|
||||
logger.warning("Observation consolidation failed for user %d", user_id, exc_info=True)
|
||||
new_summary = ""
|
||||
|
||||
if new_summary:
|
||||
cutoff = (date.today() - timedelta(days=30)).isoformat()
|
||||
async with async_session() as session:
|
||||
result = await session.execute(
|
||||
select(UserProfile).where(UserProfile.user_id == user_id)
|
||||
)
|
||||
profile = result.scalar_one_or_none()
|
||||
if profile:
|
||||
profile.learned_summary = new_summary
|
||||
profile.observations_raw = [
|
||||
o for o in (profile.observations_raw or [])
|
||||
if o.get("date", "") >= cutoff
|
||||
]
|
||||
await session.commit()
|
||||
logger.info("Consolidated observations for user %d", user_id)
|
||||
|
||||
return new_summary
|
||||
|
||||
|
||||
async def build_profile_context(user_id: int) -> str:
|
||||
"""
|
||||
Build a formatted context string from the user's structured profile
|
||||
for injection into LLM system prompts (briefing and chat).
|
||||
Returns an empty string if no meaningful data is set.
|
||||
"""
|
||||
profile = await get_profile(user_id)
|
||||
|
||||
parts: list[str] = []
|
||||
|
||||
if profile.display_name:
|
||||
parts.append(f"User's name: {profile.display_name}")
|
||||
if profile.job_title or profile.industry:
|
||||
job = " in ".join(filter(None, [profile.job_title, profile.industry]))
|
||||
parts.append(f"Occupation: {job}")
|
||||
if profile.expertise_level and profile.expertise_level != "intermediate":
|
||||
parts.append(
|
||||
f"Expertise level: {profile.expertise_level} — calibrate explanation depth accordingly"
|
||||
)
|
||||
if profile.response_style or profile.tone:
|
||||
style = profile.response_style or "balanced"
|
||||
tone = profile.tone or "casual"
|
||||
parts.append(f"Preferred response style: {style}, tone: {tone}")
|
||||
if profile.interests:
|
||||
parts.append(f"Interests: {', '.join(profile.interests)}")
|
||||
if profile.work_schedule:
|
||||
sched = profile.work_schedule
|
||||
days = ", ".join(sched.get("days") or []) or "weekdays"
|
||||
start = sched.get("start", "9:00")
|
||||
end = sched.get("end", "17:00")
|
||||
parts.append(f"Work schedule: {days}, {start}–{end}")
|
||||
if profile.learned_summary:
|
||||
parts.append(f"What the assistant has learned about this user: {profile.learned_summary}")
|
||||
|
||||
return "\n".join(parts)
|
||||
|
||||
Reference in New Issue
Block a user