refactor: Phase 8 — backend deletion (chat / voice / push / journal / curator)

Mega-commit. Strips all server-side LLM machinery now that Phase 7 has
removed the corresponding UI surfaces and the MCP HTTP endpoint is the
sole assistant interface.

Deleted (services/):
  chat, generation_buffer, generation_log, generation_task, llm, tools/
  (entire package), stt, tts, voice_config, voice_library, push,
  journal_closeout, journal_pipeline, journal_prep, journal_scheduler,
  journal_search, curator, curator_scheduler, consolidation,
  tag_suggestions, research, weather, article_fetcher, pending_actions,
  moments, assist, wikipedia.

Deleted (routes/):
  chat, voice, push, journal, quick_capture, fable_mcp_dist.

Deleted (models/):
  conversation, generation_tool_log, push_subscription,
  pending_curator_action, moment, weather_cache.

Deleted (tests/):
  test_generation_log, test_journal_*, test_consolidation, test_lookup_tool,
  test_notes_consolidation_trigger, test_record_moment_guards,
  test_research_pipeline, test_tools_*, test_tool_use_fixes,
  test_voice_library, test_weather_service, test_calendar_tool_tz,
  test_wikipedia.

Deleted (top-level):
  fable-mcp/ (legacy standalone stdio package — wheel-build pipeline
  also removed from Dockerfile).

app.py:
  - blueprint registrations for the 6 deleted routes
  - startup hook trimmed: no more Ollama warmup, KV-cache priming,
    journal/curator schedulers, voice model loading
  - shutdown hook simplified
  - httpx import dropped (was for Ollama calls)

pyproject.toml:
  - removed deps: pywebpush, feedparser, html2text, trafilatura
  - removed [voice] extras entirely
  - description updated for the MCP-first architecture

Dockerfile:
  - removed faster-whisper / piper-tts install steps
  - removed bundled piper voice download stage
  - removed fable-mcp wheel build stage

Surviving-file edits:
  - services/auth.py: drop Conversation table claim on first-user setup
  - services/backup.py: drop conversation / push-subscription export+restore;
    v1/v2 restore now silently skip pre-pivot conversation data
  - services/notes.py: drop maybe_consolidate trigger on task done/cancelled;
    drop _maybe_trigger_project_summary (LLM auto-summary)
  - services/projects.py: drop generate_project_summary + backfill_project_summaries
    (both LLM-driven)
  - services/user_profile.py: drop append_observations / consolidate /
    clear_learned_data (curator-tied) and build_profile_context
    (was LLM system-prompt builder)
  - services/notifications.py: stub out _fire_push_notif (was send_push_notification)
  - services/event_scheduler.py: drop event-reminder push + chat-retention
    cleanup job; keep CalDAV pull-sync + reminders job (in-app)
  - services/diagnostics.py: _curator_busy() always False
  - routes/notes.py: drop /assist, /assist/stream, /suggest-tags endpoints
  - routes/tasks.py: drop /<id>/consolidate endpoint
  - routes/settings.py: drop /models, KV-cache-prime-on-save, journal-schedule
    timezone hook, and the SearXNG search-test endpoint; inline _is_private_url
    (was in services/llm.py)
  - routes/admin.py: drop /voice, /voice/reload endpoints
  - routes/profile.py: drop /consolidate, /observations (GET, DELETE)
  - models/__init__.py: drop the 6 dead model imports

Frontend cascade:
  - stores/push.ts: deleted entirely (no callers after Phase 7)
  - stores/settings.ts: drop checkVoiceStatus + voice-status state
  - views/SettingsView.vue: drop Locations section + journalConfig state
    (was tied to /api/journal/config); drop JournalConfig + journal/voice
    api/client imports
  - frontend/api/client.ts: orphaned voice/journal/profile-observation/
    fable-mcp-dist exports are left as dead but harmless (call them and
    they 404; type-check is clean).

Pre-existing v1 backups that contained conversations/messages still
restore — those tables are silently dropped from the import path.
Anyone pulling the new image with a populated database will need the
Phase 9 migration to drop the dead tables (coming next).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-05-27 17:47:18 -04:00
parent 8bec68abc0
commit 91bafb641f
123 changed files with 161 additions and 19312 deletions
-13
View File
@@ -20,7 +20,6 @@ from fabledassistant.models.base import CreatedAtMixin, TimestampMixin # noqa:
from fabledassistant.models.note import Note, TaskPriority, TaskStatus # noqa: E402, F401
from fabledassistant.models.conversation import Conversation, Message # noqa: E402, F401
from fabledassistant.models.setting import Setting # noqa: E402, F401
from fabledassistant.models.user import User # noqa: E402, F401
from fabledassistant.models.app_log import AppLog # noqa: E402, F401
@@ -29,7 +28,6 @@ from fabledassistant.models.invitation import InvitationToken # noqa: E402, F40
from fabledassistant.models.embedding import NoteEmbedding # noqa: E402, F401
from fabledassistant.models.image_cache import ImageCache # noqa: E402, F401
from fabledassistant.models.project import Project # noqa: E402, F401
from fabledassistant.models.push_subscription import PushSubscription # noqa: E402, F401
from fabledassistant.models.event import Event # noqa: E402, F401
from fabledassistant.models.milestone import Milestone # noqa: E402, F401
from fabledassistant.models.task_log import TaskLog # noqa: E402, F401
@@ -38,16 +36,5 @@ from fabledassistant.models.note_version import NoteVersion # noqa: E402, F401
from fabledassistant.models.group import Group, GroupMembership # noqa: E402, F401
from fabledassistant.models.share import NoteShare, ProjectShare # noqa: E402, F401
from fabledassistant.models.notification import Notification # noqa: E402, F401
from fabledassistant.models.weather_cache import WeatherCache # noqa: E402, F401
from fabledassistant.models.api_key import ApiKey # noqa: E402, F401
from fabledassistant.models.user_profile import UserProfile # noqa: E402, F401
from fabledassistant.models.moment import ( # noqa: E402, F401
Moment,
MomentEmbedding,
moment_people,
moment_places,
moment_tasks,
moment_notes,
)
from fabledassistant.models.generation_tool_log import GenerationToolLog # noqa: E402, F401
from fabledassistant.models.pending_curator_action import PendingCuratorAction # noqa: E402, F401
-106
View File
@@ -1,106 +0,0 @@
import datetime
from sqlalchemy import Date, DateTime, ForeignKey, Index, Integer, Text
from sqlalchemy import inspect
from sqlalchemy.dialects.postgresql import JSONB
from sqlalchemy.orm import Mapped, mapped_column, relationship
from fabledassistant.models import Base
from fabledassistant.models.base import CreatedAtMixin, TimestampMixin
class Conversation(Base, TimestampMixin):
__tablename__ = "conversations"
id: Mapped[int] = mapped_column(primary_key=True)
user_id: Mapped[int | None] = mapped_column(
Integer, ForeignKey("users.id", ondelete="CASCADE"), nullable=True
)
title: Mapped[str] = mapped_column(Text, default="")
model: Mapped[str] = mapped_column(Text, default="")
# 'chat' (default) or 'journal'. Journal conversations are day-anchored
# and rendered by the /journal view; they are hidden from the main chat list.
conversation_type: Mapped[str] = mapped_column(Text, default="chat", server_default="chat")
# For journal conversations only: the calendar date this conversation covers.
day_date: Mapped[datetime.date | None] = mapped_column(Date, nullable=True)
# NULL = orphan notes only; -1 = all notes; positive int = specific project
rag_project_id: Mapped[int | None] = mapped_column(Integer, nullable=True, default=None)
# Curator scheduler bookkeeping (Phase 2, Fable #172). NULL = never run;
# scheduler processes journal conversations where this is NULL OR older
# than the most recent user message. See services/curator_scheduler.py.
last_curator_run_at: Mapped[datetime.datetime | None] = mapped_column(
DateTime(timezone=True), nullable=True, default=None
)
# Curator's most recent one-line summary of what was captured (Phase 3,
# Fable #172). Injected into the next chat turn's system prompt so the
# chat model stays aware of recent topics without needing tools to
# retrieve. ≤ 240 chars enforced by curator service. Last-write-wins;
# we don't keep history of prior summaries.
curator_summary: Mapped[str | None] = mapped_column(Text, nullable=True, default=None)
messages: Mapped[list["Message"]] = relationship(
back_populates="conversation",
cascade="all, delete-orphan",
order_by="Message.created_at",
)
__table_args__ = (
Index("ix_conversations_updated_at", "updated_at"),
Index("ix_conversations_user_id", "user_id"),
)
def to_dict(self) -> dict:
# Use loaded messages if available, otherwise 0
if "messages" in inspect(self).dict:
msg_count = len(self.messages) if self.messages else 0
else:
msg_count = 0
return {
"id": self.id,
"title": self.title,
"model": self.model,
"conversation_type": self.conversation_type,
"day_date": self.day_date.isoformat() if self.day_date else None,
"rag_project_id": self.rag_project_id,
"message_count": msg_count,
"created_at": self.created_at.isoformat(),
"updated_at": self.updated_at.isoformat(),
}
class Message(Base, CreatedAtMixin):
__tablename__ = "messages"
id: Mapped[int] = mapped_column(primary_key=True)
conversation_id: Mapped[int] = mapped_column(
Integer, ForeignKey("conversations.id", ondelete="CASCADE")
)
role: Mapped[str] = mapped_column(Text)
content: Mapped[str] = mapped_column(Text, default="")
status: Mapped[str] = mapped_column(Text, default="complete")
context_note_id: Mapped[int | None] = mapped_column(
Integer, ForeignKey("notes.id", ondelete="SET NULL"), nullable=True
)
tool_calls: Mapped[list | None] = mapped_column(JSONB, nullable=True)
# 'metadata' is reserved by SQLAlchemy Declarative — use msg_metadata as the
# Python attribute name, mapped to the 'metadata' DB column.
msg_metadata: Mapped[dict | None] = mapped_column("metadata", JSONB, nullable=True)
conversation: Mapped["Conversation"] = relationship(back_populates="messages")
__table_args__ = (
Index("ix_messages_conversation_id", "conversation_id"),
)
def to_dict(self) -> dict:
return {
"id": self.id,
"conversation_id": self.conversation_id,
"role": self.role,
"content": self.content,
"status": self.status,
"context_note_id": self.context_note_id,
"tool_calls": self.tool_calls,
"metadata": self.msg_metadata,
"created_at": self.created_at.isoformat(),
}
@@ -1,75 +0,0 @@
from datetime import datetime, timezone
from sqlalchemy import Boolean, DateTime, ForeignKey, Index, Integer, Text
from sqlalchemy.dialects.postgresql import ARRAY, JSONB
from sqlalchemy.orm import Mapped, mapped_column
from fabledassistant.models import Base
class GenerationToolLog(Base):
"""One row per assistant turn — what tools the model could/did use.
Lets the operator answer "does model X actually fire record_moment?"
empirically across model swaps without relying on anecdote.
"""
__tablename__ = "generation_tool_log"
id: Mapped[int] = mapped_column(primary_key=True)
user_id: Mapped[int] = mapped_column(
Integer,
ForeignKey("users.id", ondelete="CASCADE"),
nullable=False,
)
conv_id: Mapped[int] = mapped_column(
Integer,
ForeignKey("conversations.id", ondelete="CASCADE"),
nullable=False,
)
# SET NULL (migration matches) so the telemetry row survives if the
# underlying assistant message is later deleted.
assistant_message_id: Mapped[int | None] = mapped_column(
Integer,
ForeignKey("messages.id", ondelete="SET NULL"),
nullable=True,
)
model: Mapped[str] = mapped_column(Text, nullable=False)
think_enabled: Mapped[bool] = mapped_column(Boolean, nullable=False)
tools_available: Mapped[list[str]] = mapped_column(
ARRAY(Text), nullable=False, default=list
)
tools_attempted: Mapped[list[str]] = mapped_column(
ARRAY(Text), nullable=False, default=list
)
tools_succeeded: Mapped[list[str]] = mapped_column(
ARRAY(Text), nullable=False, default=list
)
# JSONB list of {name, error} dicts.
tools_failed: Mapped[list[dict]] = mapped_column(
JSONB, nullable=False, default=list
)
created_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True),
default=lambda: datetime.now(timezone.utc),
)
__table_args__ = (
Index("ix_generation_tool_log_user_created", "user_id", created_at.desc()),
Index("ix_generation_tool_log_conv", "conv_id"),
)
def to_dict(self) -> dict:
return {
"id": self.id,
"user_id": self.user_id,
"conv_id": self.conv_id,
"assistant_message_id": self.assistant_message_id,
"model": self.model,
"think_enabled": self.think_enabled,
"tools_available": list(self.tools_available or []),
"tools_attempted": list(self.tools_attempted or []),
"tools_succeeded": list(self.tools_succeeded or []),
"tools_failed": list(self.tools_failed or []),
"created_at": self.created_at.isoformat() if self.created_at else None,
}
-130
View File
@@ -1,130 +0,0 @@
import datetime
from sqlalchemy import Boolean, Column, Date, DateTime, ForeignKey, Index, Integer, Table, Text
from sqlalchemy.dialects.postgresql import ARRAY, JSONB
from sqlalchemy.orm import Mapped, mapped_column, relationship
from fabledassistant.models import Base
# Junction tables. People, Places, Tasks, and (regular) Notes all live in
# the `notes` table — the four junctions are kept separate (rather than one
# merged table with a discriminator) so per-kind queries don't require a
# filter, and so the schema is explicit about which links are which.
moment_people = Table(
"moment_people",
Base.metadata,
Column("moment_id", Integer, ForeignKey("moments.id", ondelete="CASCADE"), primary_key=True),
Column("person_id", Integer, ForeignKey("notes.id", ondelete="CASCADE"), primary_key=True),
)
moment_places = Table(
"moment_places",
Base.metadata,
Column("moment_id", Integer, ForeignKey("moments.id", ondelete="CASCADE"), primary_key=True),
Column("place_id", Integer, ForeignKey("notes.id", ondelete="CASCADE"), primary_key=True),
)
moment_tasks = Table(
"moment_tasks",
Base.metadata,
Column("moment_id", Integer, ForeignKey("moments.id", ondelete="CASCADE"), primary_key=True),
Column("task_id", Integer, ForeignKey("notes.id", ondelete="CASCADE"), primary_key=True),
)
moment_notes = Table(
"moment_notes",
Base.metadata,
Column("moment_id", Integer, ForeignKey("moments.id", ondelete="CASCADE"), primary_key=True),
Column("note_id", Integer, ForeignKey("notes.id", ondelete="CASCADE"), primary_key=True),
)
class Moment(Base):
"""A small structured extraction from a journal conversation.
Many per day. Emitted by the LLM via the `record_moment` tool when it
notices a meaningful beat. Stored separately from Notes — they are
different in kind: Notes are curated artifacts; Moments are ambient
trace data with their own embedding index, so notes-RAG and journal-RAG
cannot cross-contaminate.
"""
__tablename__ = "moments"
id: Mapped[int] = mapped_column(Integer, primary_key=True)
user_id: Mapped[int] = mapped_column(
Integer, ForeignKey("users.id", ondelete="CASCADE"), nullable=False
)
conversation_id: Mapped[int | None] = mapped_column(
Integer, ForeignKey("conversations.id", ondelete="SET NULL"), nullable=True
)
source_message_id: Mapped[int | None] = mapped_column(
Integer, ForeignKey("messages.id", ondelete="SET NULL"), nullable=True
)
day_date: Mapped[datetime.date] = mapped_column(Date, nullable=False)
occurred_at: Mapped[datetime.datetime] = mapped_column(
DateTime(timezone=True), nullable=False
)
recorded_at: Mapped[datetime.datetime] = mapped_column(
DateTime(timezone=True),
nullable=False,
default=lambda: datetime.datetime.now(datetime.timezone.utc),
)
content: Mapped[str] = mapped_column(Text, nullable=False)
raw_excerpt: Mapped[str | None] = mapped_column(Text, nullable=True)
tags: Mapped[list[str]] = mapped_column(ARRAY(Text), nullable=False, default=list)
pinned: Mapped[bool] = mapped_column(Boolean, nullable=False, default=False)
people = relationship("Note", secondary=moment_people, lazy="selectin", viewonly=True)
places = relationship("Note", secondary=moment_places, lazy="selectin", viewonly=True)
tasks = relationship("Note", secondary=moment_tasks, lazy="selectin", viewonly=True)
notes = relationship("Note", secondary=moment_notes, lazy="selectin", viewonly=True)
__table_args__ = (
Index("ix_moments_user_day", "user_id", "day_date"),
Index("ix_moments_user_occurred", "user_id", "occurred_at"),
)
def to_dict(self, *, include_links: bool = True) -> dict:
result: dict = {
"id": self.id,
"user_id": self.user_id,
"conversation_id": self.conversation_id,
"source_message_id": self.source_message_id,
"day_date": self.day_date.isoformat(),
"occurred_at": self.occurred_at.isoformat(),
"recorded_at": self.recorded_at.isoformat(),
"content": self.content,
"raw_excerpt": self.raw_excerpt,
"tags": list(self.tags or []),
"pinned": self.pinned,
}
if include_links:
result["people"] = [{"id": p.id, "title": p.title} for p in (self.people or [])]
result["places"] = [{"id": p.id, "title": p.title} for p in (self.places or [])]
result["task_ids"] = [t.id for t in (self.tasks or [])]
result["note_ids"] = [n.id for n in (self.notes or [])]
return result
class MomentEmbedding(Base):
"""Embedding vector for a Moment — used by `search_journal` semantic mode.
Stored separately from `note_embeddings` so notes-RAG and journal-RAG
cannot cross-contaminate. This is a hard invariant of the journal design.
"""
__tablename__ = "moment_embeddings"
moment_id: Mapped[int] = mapped_column(
Integer,
ForeignKey("moments.id", ondelete="CASCADE"),
primary_key=True,
)
user_id: Mapped[int] = mapped_column(Integer, nullable=False, index=True)
embedding: Mapped[list] = mapped_column(JSONB, nullable=False)
updated_at: Mapped[datetime.datetime] = mapped_column(
DateTime(timezone=True),
default=lambda: datetime.datetime.now(datetime.timezone.utc),
)
@@ -1,81 +0,0 @@
from datetime import datetime, timezone
from sqlalchemy import Boolean, DateTime, ForeignKey, Index, Integer, Text
from sqlalchemy.dialects.postgresql import JSONB
from sqlalchemy.orm import Mapped, mapped_column
from fabledassistant.models import Base
class PendingCuratorAction(Base):
"""Curator-proposed mutation awaiting user approval.
The curator can be confidently wrong, and the user is not in the loop
when it runs. Additive operations land directly; mutating operations
(update_*, delete_*) write a row here instead. The user reviews each
one from the journal's Needs Review panel.
On approval, the original tool call is replayed via execute_tool with
authority="user" — bypassing the curator interceptor so the request
just runs.
"""
__tablename__ = "pending_curator_actions"
id: Mapped[int] = mapped_column(primary_key=True)
user_id: Mapped[int] = mapped_column(
Integer,
ForeignKey("users.id", ondelete="CASCADE"),
nullable=False,
)
# SET NULL so a curator-proposed action survives if its source
# conversation is later deleted — the user might still want to
# review and approve it.
conv_id: Mapped[int | None] = mapped_column(
Integer,
ForeignKey("conversations.id", ondelete="SET NULL"),
nullable=True,
)
# The tool name the curator wanted to call (`update_task`, `delete_note`,
# etc.). Used at approval-replay time to dispatch through execute_tool.
action_type: Mapped[str] = mapped_column(Text, nullable=False)
# Display hints — what the UI shows in the card header.
target_type: Mapped[str | None] = mapped_column(Text, nullable=True)
target_id: Mapped[int | None] = mapped_column(Integer, nullable=True)
target_label: Mapped[str | None] = mapped_column(Text, nullable=True)
# The curator's proposed arguments — replayed verbatim on approval.
payload: Mapped[dict] = mapped_column(JSONB, nullable=False, default=dict)
# State of the target at proposal time. Lets the UI render an honest
# diff (current → proposed) even if other work modifies the entity
# between proposal and review.
current_snapshot: Mapped[dict] = mapped_column(
JSONB, nullable=False, default=dict
)
status: Mapped[str] = mapped_column(
Text, nullable=False, default="pending", server_default="pending"
)
created_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True),
default=lambda: datetime.now(timezone.utc),
)
reviewed_at: Mapped[datetime | None] = mapped_column(
DateTime(timezone=True), nullable=True, default=None
)
def to_dict(self) -> dict:
return {
"id": self.id,
"user_id": self.user_id,
"conv_id": self.conv_id,
"action_type": self.action_type,
"target_type": self.target_type,
"target_id": self.target_id,
"target_label": self.target_label,
"payload": dict(self.payload or {}),
"current_snapshot": dict(self.current_snapshot or {}),
"status": self.status,
"created_at": self.created_at.isoformat() if self.created_at else None,
"reviewed_at": (
self.reviewed_at.isoformat() if self.reviewed_at else None
),
}
@@ -1,20 +0,0 @@
from datetime import datetime
from sqlalchemy import DateTime, ForeignKey, Integer, Text, UniqueConstraint
from sqlalchemy.orm import Mapped, mapped_column
from fabledassistant.models import Base
from fabledassistant.models.base import CreatedAtMixin
class PushSubscription(Base, CreatedAtMixin):
__tablename__ = "push_subscriptions"
id: Mapped[int] = mapped_column(primary_key=True)
user_id: Mapped[int] = mapped_column(Integer, ForeignKey("users.id", ondelete="CASCADE"))
endpoint: Mapped[str] = mapped_column(Text)
p256dh: Mapped[str] = mapped_column(Text)
auth: Mapped[str] = mapped_column(Text)
user_agent: Mapped[str | None] = mapped_column(Text, nullable=True)
last_used: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True)
__table_args__ = (
UniqueConstraint("user_id", "endpoint", name="uq_push_subscription_user_endpoint"),
)
@@ -1,38 +0,0 @@
from datetime import datetime, timezone
from sqlalchemy import DateTime, ForeignKey, Index, Integer, Text, UniqueConstraint
from sqlalchemy.dialects.postgresql import JSONB
from sqlalchemy.orm import Mapped, mapped_column
from fabledassistant.models import Base
class WeatherCache(Base):
__tablename__ = "weather_cache"
id: Mapped[int] = mapped_column(primary_key=True)
user_id: Mapped[int] = mapped_column(Integer, ForeignKey("users.id", ondelete="CASCADE"))
# Unique key per location, e.g. "home", "work", or "event:<uid>"
location_key: Mapped[str] = mapped_column(Text)
location_label: Mapped[str] = mapped_column(Text, default="")
# Current 7-day forecast from Open-Meteo (raw JSON)
forecast_json: Mapped[dict | None] = mapped_column(JSONB, nullable=True)
# Previous forecast — used to detect changes between fetches
previous_json: Mapped[dict | None] = mapped_column(JSONB, nullable=True)
fetched_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True), default=lambda: datetime.now(timezone.utc)
)
__table_args__ = (
UniqueConstraint("user_id", "location_key", name="uq_weather_cache_user_location"),
Index("ix_weather_cache_user_id", "user_id"),
)
def to_dict(self) -> dict:
return {
"id": self.id,
"location_key": self.location_key,
"location_label": self.location_label,
"forecast_json": self.forecast_json,
"fetched_at": self.fetched_at.isoformat(),
}