feat: voice S2S — faster-whisper STT, Kokoro TTS, PTT overlay
Implements full speech-to-speech pipeline (all 4 phases): Backend (Phase 1): - services/stt.py: lazy WhisperModel singleton, run_in_executor transcription - services/tts.py: lazy KPipeline singleton, WAV synthesis at 24kHz/16-bit - routes/voice.py: /api/voice/status, /voices, /transcribe, /synthesise - config.py: VOICE_ENABLED, STT_BACKEND, STT_MODEL, TTS_BACKEND env vars - app.py: load STT/TTS models at startup when VOICE_ENABLED=true - llm.py: voice_mode + voice_speech_style params inject speak-naturally prefix - generation_task.py: voice_mode passed through from chat route - chat.py: "voice" conversation type allowed + excluded from retention cleanup - pyproject.toml + Dockerfile: faster-whisper, kokoro, soundfile dependencies Frontend (Phases 2–4): - composables/useVoiceRecorder.ts: MediaRecorder PTT wrapper - composables/useVoiceAudio.ts: AudioContext WAV playback wrapper - BriefingView.vue: Listen button (TTS read-aloud), auto-TTS mode, mic PTT - VoiceOverlay.vue: global floating PTT button; creates/reuses voice conv; full record→transcribe→stream→TTS flow; Space bar hold-to-talk via App.vue - SettingsView.vue: Voice tab (status badge, speech style, voice/speed) - App.vue: mounts VoiceOverlay; Space keydown/keyup fires voice:ptt-toggle - api/client.ts: getVoiceStatus, getVoiceList, transcribeAudio, synthesiseSpeech Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
@@ -31,6 +31,7 @@ from fabledassistant.routes.users import users_bp
|
||||
from fabledassistant.routes.api_keys import api_keys_bp
|
||||
from fabledassistant.routes.events import events_bp
|
||||
from fabledassistant.routes.search import search_bp
|
||||
from fabledassistant.routes.voice import voice_bp
|
||||
|
||||
STATIC_DIR = Path(__file__).parent / "static"
|
||||
logger = logging.getLogger(__name__)
|
||||
@@ -92,6 +93,7 @@ def create_app() -> Quart:
|
||||
app.register_blueprint(api_keys_bp)
|
||||
app.register_blueprint(events_bp)
|
||||
app.register_blueprint(search_bp)
|
||||
app.register_blueprint(voice_bp)
|
||||
|
||||
@app.before_request
|
||||
async def before_request():
|
||||
@@ -264,6 +266,13 @@ def create_app() -> Quart:
|
||||
from fabledassistant.services.briefing_scheduler import start_briefing_scheduler
|
||||
await start_briefing_scheduler(asyncio.get_running_loop())
|
||||
|
||||
# Voice model loading (only when feature is enabled)
|
||||
if Config.VOICE_ENABLED:
|
||||
from fabledassistant.services.stt import load_stt_model
|
||||
from fabledassistant.services.tts import load_tts_model
|
||||
asyncio.create_task(load_stt_model())
|
||||
asyncio.create_task(load_tts_model())
|
||||
|
||||
@app.after_serving
|
||||
async def shutdown():
|
||||
from fabledassistant.services.briefing_scheduler import stop_briefing_scheduler
|
||||
|
||||
Reference in New Issue
Block a user