feat: voice S2S — faster-whisper STT, Kokoro TTS, PTT overlay

Implements full speech-to-speech pipeline (all 4 phases):

Backend (Phase 1):
- services/stt.py: lazy WhisperModel singleton, run_in_executor transcription
- services/tts.py: lazy KPipeline singleton, WAV synthesis at 24kHz/16-bit
- routes/voice.py: /api/voice/status, /voices, /transcribe, /synthesise
- config.py: VOICE_ENABLED, STT_BACKEND, STT_MODEL, TTS_BACKEND env vars
- app.py: load STT/TTS models at startup when VOICE_ENABLED=true
- llm.py: voice_mode + voice_speech_style params inject speak-naturally prefix
- generation_task.py: voice_mode passed through from chat route
- chat.py: "voice" conversation type allowed + excluded from retention cleanup
- pyproject.toml + Dockerfile: faster-whisper, kokoro, soundfile dependencies

Frontend (Phases 2–4):
- composables/useVoiceRecorder.ts: MediaRecorder PTT wrapper
- composables/useVoiceAudio.ts: AudioContext WAV playback wrapper
- BriefingView.vue: Listen button (TTS read-aloud), auto-TTS mode, mic PTT
- VoiceOverlay.vue: global floating PTT button; creates/reuses voice conv;
  full record→transcribe→stream→TTS flow; Space bar hold-to-talk via App.vue
- SettingsView.vue: Voice tab (status badge, speech style, voice/speed)
- App.vue: mounts VoiceOverlay; Space keydown/keyup fires voice:ptt-toggle
- api/client.ts: getVoiceStatus, getVoiceList, transcribeAudio, synthesiseSpeech

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
2026-03-29 20:03:38 -04:00
parent 3581cc1582
commit 6f84d90dff
18 changed files with 1524 additions and 6 deletions
@@ -152,6 +152,7 @@ async def run_generation(
rag_project_id: int | None = None,
workspace_project_id: int | None = None,
user_timezone: str | None = None,
voice_mode: bool = False,
) -> None:
"""Stream LLM response into buffer with periodic DB flushes."""
MAX_TOOL_ROUNDS = 5
@@ -177,6 +178,12 @@ async def run_generation(
# Phase 3: Build context and wait for model in parallel.
model_load_task = asyncio.create_task(wait_for_model_loaded(model, timeout=180.0))
# Fetch voice_speech_style from user settings when voice_mode is active.
voice_speech_style = "conversational"
if voice_mode:
from fabledassistant.services.settings import get_setting
voice_speech_style = await get_setting(user_id, "voice_speech_style", "conversational")
context_task = asyncio.create_task(build_context(
user_id, history_to_use, context_note_id, user_content,
history_summary=history_summary,
@@ -186,6 +193,8 @@ async def run_generation(
workspace_project_id=workspace_project_id,
user_timezone=user_timezone,
conv_id=conv_id,
voice_mode=voice_mode,
voice_speech_style=voice_speech_style,
))
messages, context_meta = await context_task