feat: voice S2S — faster-whisper STT, Kokoro TTS, PTT overlay

Implements full speech-to-speech pipeline (all 4 phases):

Backend (Phase 1):
- services/stt.py: lazy WhisperModel singleton, run_in_executor transcription
- services/tts.py: lazy KPipeline singleton, WAV synthesis at 24kHz/16-bit
- routes/voice.py: /api/voice/status, /voices, /transcribe, /synthesise
- config.py: VOICE_ENABLED, STT_BACKEND, STT_MODEL, TTS_BACKEND env vars
- app.py: load STT/TTS models at startup when VOICE_ENABLED=true
- llm.py: voice_mode + voice_speech_style params inject speak-naturally prefix
- generation_task.py: voice_mode passed through from chat route
- chat.py: "voice" conversation type allowed + excluded from retention cleanup
- pyproject.toml + Dockerfile: faster-whisper, kokoro, soundfile dependencies

Frontend (Phases 2–4):
- composables/useVoiceRecorder.ts: MediaRecorder PTT wrapper
- composables/useVoiceAudio.ts: AudioContext WAV playback wrapper
- BriefingView.vue: Listen button (TTS read-aloud), auto-TTS mode, mic PTT
- VoiceOverlay.vue: global floating PTT button; creates/reuses voice conv;
  full record→transcribe→stream→TTS flow; Space bar hold-to-talk via App.vue
- SettingsView.vue: Voice tab (status badge, speech style, voice/speed)
- App.vue: mounts VoiceOverlay; Space keydown/keyup fires voice:ptt-toggle
- api/client.ts: getVoiceStatus, getVoiceList, transcribeAudio, synthesiseSpeech

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
2026-03-29 20:03:38 -04:00
parent 3581cc1582
commit 6f84d90dff
18 changed files with 1524 additions and 6 deletions
+48
View File
@@ -619,3 +619,51 @@ export function getNewsItems(params: GetNewsItemsParams = {}) {
`/api/briefing/news?${p}`
)
}
// ─── Voice ────────────────────────────────────────────────────────────────────
export interface VoiceStatusResult {
enabled: boolean
stt: boolean
tts: boolean
stt_model?: string
tts_backend?: string
}
export interface VoiceEntry {
id: string
label: string
}
export const getVoiceStatus = () => apiGet<VoiceStatusResult>('/api/voice/status')
export const getVoiceList = () =>
apiGet<{ voices: VoiceEntry[] }>('/api/voice/voices').then(r => r.voices)
export async function transcribeAudio(blob: Blob): Promise<{ transcript: string; duration_ms: number }> {
const form = new FormData()
form.append('audio', blob, 'audio.webm')
const res = await fetch('/api/voice/transcribe', { method: 'POST', body: form })
if (!res.ok) {
const err = await res.json().catch(() => ({ error: `HTTP ${res.status}` }))
throw new ApiError(res.status, err)
}
return res.json()
}
export async function synthesiseSpeech(
text: string,
voice?: string,
speed?: number
): Promise<Blob> {
const res = await fetch('/api/voice/synthesise', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ text, voice: voice ?? 'af_heart', speed: speed ?? 1.0 }),
})
if (!res.ok) {
const err = await res.json().catch(() => ({ error: `HTTP ${res.status}` }))
throw new ApiError(res.status, err)
}
return res.blob()
}