bvandeusen/FabledScribe
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 50 Wiki Activity

feat(voice): swap kokoro TTS → piper-tts

Browse Source 
Kokoro has been stale upstream since April 2025 (`requires_python<3.13`),
which broke the Python 3.14 build. Piper is the active replacement:
maintained by OHF/Home Assistant, depends only on onnxruntime +
pathvalidate (no torch, no spacy, no transformers), and has cp314
support today.

Dockerfile:
- Add `pip install piper-tts` after the STT install.
- Bundle two default voices (en_US-amy-medium, en_US-ryan-medium) into
  /opt/piper-voices at build. Additional voices can be downloaded into
  /data/voices via the admin UI (separate commit).
- Image add over the STT-only baseline: ~150 MB.

services/tts.py — full rewrite:
- New voice-discovery layer scans /opt/piper-voices + /data/voices for
  .onnx + .onnx.json pairs. /data wins over /opt for the same id so
  admin-downloaded voices can override bundled defaults.
- Single PiperVoice kept warm; switches via _switch_voice() when the
  user changes their voice_tts_voice setting.
- list_voices() returns metadata read from .onnx.json sidecars (label
  derived from filename, language, quality, sample_rate).
- synthesise() uses piper's SynthesisConfig; converts kokoro-shaped
  `speed` multiplier to piper's `length_scale` (1.0 / speed).
- `voice_blend` parameter accepted but ignored — piper has no blend
  equivalent; first entry's voice is used if anything is passed.
- Dropped: HuggingFace commit-hash tracking (~80 lines), the daily
  check_for_kokoro_updates task, voice-tensor blending math.

routes/voice.py:
- tts_backend reports "piper" in /api/voice/status.
- /api/voice/voices no longer requires tts_available() — even with
  the active voice failed to load, the catalog still lets the user
  pick a different one.
- Synthesise request body dropped the voice_blend field; speed and
  voice still supported.

alembic 0047_reset_voice_tts_settings:
- Deletes any stored voice_tts_voice (kokoro IDs that don't map to
  piper) and voice_tts_blend (no piper equivalent) rows. Both
  re-default cleanly on next read.

frontend:
- VoiceBlendEntry type removed from api/client.ts.
- synthesiseSpeech() signature dropped the voiceBlend parameter.
- SettingsView.vue Voice Blend section removed entirely (slider,
  preview, slot management). voice_tts_blend save path removed.
- Default voice id changed from "af_heart" to "en_US-amy-medium".
- VoiceEntry gains optional language/quality/sample_rate fields
  from the richer piper sidecar metadata.

Voice paths remain lazily guarded — `VOICE_ENABLED=false` (default)
starts the app cleanly regardless of which TTS deps are present.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Bryan Van Deusen
2026-05-22 07:59:09 -04:00
parent c91b9c46ff
commit a28f75994a
6 changed files with 317 additions and 363 deletions
Download Patch File Download Diff File Expand all files Collapse all files
frontend/src/api/client.ts
+10 -13
View File
@@ -623,11 +623,9 @@ export interface VoiceStatusResult {
export interface VoiceEntry {
id: string
label: string
}
export interface VoiceBlendEntry {
voice: string
weight: number
language?: string
quality?: string
sample_rate?: number
}
export const getVoiceStatus = () => apiGet<VoiceStatusResult>('/api/voice/status')
@@ -651,16 +649,15 @@ export async function synthesiseSpeech(
text: string,
voice?: string,
speed?: number,
voiceBlend?: VoiceBlendEntry[]
): Promise<Blob> {
// Only send voice/speed/blend when explicitly provided — omitting them lets
// the server auto-load the user's saved voice settings (voice, speed, blend).
// Only send voice/speed when explicitly provided — omitting them lets the
// server auto-load the user's saved voice settings.
// Voice blending was removed with the kokoro → piper migration (piper has
// no blend equivalent); callers that previously passed `voiceBlend` should
// pass the first voice's id as `voice` instead.
const body: Record<string, unknown> = { text }
if (voiceBlend && voiceBlend.length >= 2) {
body.voice_blend = voiceBlend
if (speed !== undefined) body.speed = speed
} else if (voice !== undefined || speed !== undefined) {
body.voice = voice ?? 'af_heart'
if (voice !== undefined || speed !== undefined) {
body.voice = voice ?? 'en_US-amy-medium'
body.speed = speed ?? 1.0
}
const res = await fetch('/api/voice/synthesise', {
frontend/src/views/SettingsView.vue
+10 -120
View File
@@ -1,10 +1,9 @@
<script setup lang="ts">
import { X } from "lucide-vue-next";
import { ref, computed, watch, onMounted } from "vue";
import { useSettingsStore } from "@/stores/settings";
import { useAuthStore } from "@/stores/auth";
import { useToastStore } from "@/stores/toast";
import { apiGet, apiPost, apiPut, apiDelete, listGroups, createGroup, deleteGroup, listGroupMembers, addGroupMember, removeGroupMember, searchUsers, getFableMcpInfo, listApiKeys, createApiKey as apiCreateApiKey, revokeApiKey as apiRevokeApiKey, getVoiceStatus, getVoiceList, synthesiseSpeech, getProfile, updateProfile, consolidateProfile, clearProfileObservations, listProfileObservations, getJournalConfig, saveJournalConfig, geocodeAddress, type ApiKeyEntry, type GroupEntry, type GroupMember, type UserSearchResult, type VoiceStatusResult, type VoiceEntry, type VoiceBlendEntry, type UserProfile, type JournalConfig, type ProfileObservationEntry } from "@/api/client";
import { apiGet, apiPost, apiPut, apiDelete, listGroups, createGroup, deleteGroup, listGroupMembers, addGroupMember, removeGroupMember, searchUsers, getFableMcpInfo, listApiKeys, createApiKey as apiCreateApiKey, revokeApiKey as apiRevokeApiKey, getVoiceStatus, getVoiceList, synthesiseSpeech, getProfile, updateProfile, consolidateProfile, clearProfileObservations, listProfileObservations, getJournalConfig, saveJournalConfig, geocodeAddress, type ApiKeyEntry, type GroupEntry, type GroupMember, type UserSearchResult, type VoiceStatusResult, type VoiceEntry, type UserProfile, type JournalConfig, type ProfileObservationEntry } from "@/api/client";
import { usePushStore } from "@/stores/push";
import type { User } from "@/types/auth";
import PaginationBar from "@/components/PaginationBar.vue";
@@ -457,58 +456,19 @@ const searchResults = ref<{ url: string; title: string; snippet: string }[]>([])
const searchLoading = ref(false);
const searchError = ref("");
// Voice settings
// Voice settings. Default voice id matches the bundled piper voice
// (en_US-amy-medium); legacy kokoro IDs in user settings were cleared
// by alembic migration 0047, so the first read after upgrade returns "".
const voiceStatus = ref<VoiceStatusResult | null>(null);
const voiceStatusLoading = ref(false);
const availableVoices = ref<VoiceEntry[]>([]);
const voiceTtsVoice = ref("af_heart");
const voiceTtsVoice = ref("en_US-amy-medium");
const voiceTtsSpeed = ref(1.0);
const voiceSpeechStyle = ref("conversational");
const savingVoice = ref(false);
const voiceSaved = ref(false);
const voiceTabLoaded = ref(false);
// Voice blend
const blendEnabled = ref(false);
const voiceBlend = ref<VoiceBlendEntry[]>([
{ voice: "af_heart", weight: 1.0 },
{ voice: "af_bella", weight: 1.0 },
]);
const blendPreviewText = ref("Hello! I'm your assistant. How can I help you today?");
const blendPreviewing = ref(false);
function addBlendSlot() {
voiceBlend.value.push({ voice: "af_heart", weight: 1.0 });
}
function removeBlendSlot(idx: number) {
if (voiceBlend.value.length > 2) voiceBlend.value.splice(idx, 1);
}
async function previewBlend() {
if (blendPreviewing.value || !blendPreviewText.value.trim()) return;
blendPreviewing.value = true;
// Create AudioContext synchronously in user-gesture context to bypass autoplay policy
const ctx = new AudioContext();
try {
const blob = await synthesiseSpeech(
blendPreviewText.value,
undefined,
voiceTtsSpeed.value,
voiceBlend.value,
);
const buf = await ctx.decodeAudioData(await blob.arrayBuffer());
const src = ctx.createBufferSource();
src.buffer = buf;
src.connect(ctx.destination);
src.start(0);
} catch {
toastStore.show("Preview failed — TTS may not be ready", "error");
} finally {
blendPreviewing.value = false;
}
}
async function loadVoiceTab() {
if (voiceTabLoaded.value) return;
voiceTabLoaded.value = true;
@@ -558,7 +518,6 @@ async function saveVoiceSettings() {
voice_tts_voice: voiceTtsVoice.value,
voice_tts_speed: String(voiceTtsSpeed.value),
voice_speech_style: voiceSpeechStyle.value,
voice_tts_blend: blendEnabled.value ? JSON.stringify(voiceBlend.value) : "",
});
voiceSaved.value = true;
setTimeout(() => { voiceSaved.value = false; }, 2000);
@@ -805,19 +764,12 @@ onMounted(async () => {
// Load journal config (locations, temp unit, prep schedule)
await loadJournalConfig();
// Load voice settings
// Load voice settings. Voice blending was removed with the kokoro→piper
// migration (piper has no blend equivalent); legacy voice_tts_blend rows
// were dropped in migration 0047.
if (allSettings.voice_tts_voice) voiceTtsVoice.value = allSettings.voice_tts_voice;
if (allSettings.voice_tts_speed) voiceTtsSpeed.value = parseFloat(allSettings.voice_tts_speed);
if (allSettings.voice_speech_style) voiceSpeechStyle.value = allSettings.voice_speech_style;
if (allSettings.voice_tts_blend) {
try {
const parsed = JSON.parse(allSettings.voice_tts_blend);
if (Array.isArray(parsed) && parsed.length >= 2) {
voiceBlend.value = parsed;
blendEnabled.value = true;
}
} catch { /* ignore malformed */ }
}
// Load CalDAV settings
try {
@@ -2407,70 +2359,8 @@ function formatUserDate(iso: string): string {
</div>
</section>
<section v-if="voiceStatus?.enabled && voiceStatus?.tts" class="settings-section full-width">
<h2>Voice Blend</h2>
<p class="section-desc">
Mix two or more voice styles by weight. The resulting blended voice is used instead of
the single voice above when enabled.
</p>
<div class="field-group">
<label class="toggle-label">
<input type="checkbox" v-model="blendEnabled" />
<span>Enable voice blending</span>
</label>
</div>
<template v-if="blendEnabled">
<div class="blend-slots">
<div v-for="(entry, idx) in voiceBlend" :key="idx" class="blend-slot">
<select class="input blend-voice-select" v-model="entry.voice">
<option v-for="v in availableVoices" :key="v.id" :value="v.id">{{ v.label }}</option>
</select>
<div class="blend-weight-row">
<input
type="range"
min="0.1"
max="2.0"
step="0.05"
v-model.number="entry.weight"
class="range-input blend-weight-slider"
/>
<span class="blend-weight-label">{{ entry.weight.toFixed(2) }}</span>
<button
v-if="voiceBlend.length > 2"
class="btn-remove-slot"
@click="removeBlendSlot(idx)"
title="Remove this voice"
><X :size="16" /></button>
</div>
</div>
</div>
<div class="blend-actions">
<button class="btn-secondary" @click="addBlendSlot" :disabled="voiceBlend.length >= 5">
+ Add voice
</button>
</div>
<div class="field-group" style="margin-top: 1rem;">
<label class="field-label">Preview text</label>
<input
type="text"
class="input"
v-model="blendPreviewText"
placeholder="Text to speak for preview…"
maxlength="300"
/>
</div>
<div class="form-actions">
<button class="btn-secondary" @click="previewBlend" :disabled="blendPreviewing">
{{ blendPreviewing ? 'Generating…' : '▶ Preview blend' }}
</button>
</div>
</template>
</section>
<!-- Voice Blend section removed with the kokoro → piper migration
(2026-05-22). Piper has no voice-blending equivalent. -->
</div>