Compare commits
8 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| f146485df3 | |||
| eaf70500b8 | |||
| 6f84d90dff | |||
| 3581cc1582 | |||
| 3dd879640a | |||
| 0cbeb6b7ac | |||
| 218f946e48 | |||
| 00643c778e |
@@ -13,6 +13,8 @@ WORKDIR /app
|
||||
COPY pyproject.toml .
|
||||
COPY src/ src/
|
||||
RUN pip install --no-cache-dir .
|
||||
# Voice dependencies (faster-whisper, Kokoro TTS, soundfile) — activated at runtime via VOICE_ENABLED
|
||||
RUN pip install --no-cache-dir faster-whisper kokoro soundfile
|
||||
|
||||
# Build the fable-mcp wheel so it can be served for download
|
||||
COPY fable-mcp/ fable-mcp/
|
||||
|
||||
+1
-1
@@ -15,7 +15,7 @@ services:
|
||||
environment:
|
||||
DATABASE_URL: "postgresql+asyncpg://${POSTGRES_USER:-fabled}:${POSTGRES_PASSWORD:-fabled}@db:5432/${POSTGRES_DB:-fabledassistant}"
|
||||
OLLAMA_URL: "http://ollama:11434"
|
||||
OLLAMA_MODEL: "${OLLAMA_MODEL:-llama3.1}"
|
||||
OLLAMA_MODEL: "${OLLAMA_MODEL:-qwen3:8B}"
|
||||
SECRET_KEY: "${SECRET_KEY:-dev-secret-change-me}"
|
||||
# Uncomment and set to enable web research and image search via SearXNG:
|
||||
# SEARXNG_URL: "http://searxng:8080"
|
||||
|
||||
@@ -0,0 +1,200 @@
|
||||
# Speech-to-Speech (S2S) Design Spec
|
||||
|
||||
**Branch:** `feature/voice-s2s`
|
||||
**Date:** 2026-03-29
|
||||
**Status:** Approved for implementation
|
||||
|
||||
---
|
||||
|
||||
## Decisions
|
||||
|
||||
- **STT:** faster-whisper (in-process Python, model size configurable via `STT_MODEL` env var)
|
||||
- **TTS:** Kokoro TTS (in-process Python, voice/speed/style configurable per-user)
|
||||
- **Input mode:** Push-to-talk (Phase 1); VAD deferred
|
||||
- **No browser STT/TTS fallbacks** — self-hosted only, data stays on-server
|
||||
- **No Android app** — web only for this implementation
|
||||
|
||||
---
|
||||
|
||||
## New Env Vars (`config.py`)
|
||||
|
||||
| Var | Default | Description |
|
||||
|---|---|---|
|
||||
| `VOICE_ENABLED` | `false` | Feature flag — opt-in |
|
||||
| `STT_BACKEND` | `faster-whisper` | Only supported value currently |
|
||||
| `STT_MODEL` | `base.en` | `tiny.en` / `base.en` / `small.en` / `medium.en` |
|
||||
| `TTS_BACKEND` | `kokoro` | Only supported value currently |
|
||||
|
||||
---
|
||||
|
||||
## Per-User Settings (stored in `settings` table)
|
||||
|
||||
| Key | Default | Description |
|
||||
|---|---|---|
|
||||
| `voice_tts_voice` | `af_heart` | Kokoro voice ID |
|
||||
| `voice_tts_speed` | `1.0` | Speech rate, 0.7–1.3 |
|
||||
| `voice_speech_style` | `conversational` | `conversational` / `concise` / `detailed` |
|
||||
|
||||
---
|
||||
|
||||
## New Backend Files
|
||||
|
||||
### `src/fabledassistant/services/stt.py`
|
||||
Lazy singleton `WhisperModel` loader. Public API:
|
||||
- `load_stt_model()` — called at startup via `asyncio.create_task`
|
||||
- `transcribe(audio_bytes, mime_type) -> str` — runs in `run_in_executor`; writes bytes to `NamedTemporaryFile`, returns concatenated segment text
|
||||
- `stt_available() -> bool`
|
||||
|
||||
### `src/fabledassistant/services/tts.py`
|
||||
Lazy singleton `KPipeline` loader. Public API:
|
||||
- `load_tts_model()` — called at startup
|
||||
- `synthesise(text, voice, speed) -> bytes` — runs in `run_in_executor`; returns WAV bytes (24kHz, 16-bit mono)
|
||||
- `list_voices() -> list[dict]` — returns static list of known Kokoro voice IDs + labels
|
||||
- `tts_available() -> bool`
|
||||
|
||||
### `src/fabledassistant/routes/voice.py`
|
||||
Blueprint at `/api/voice`, all routes `@login_required`.
|
||||
|
||||
| Endpoint | Method | Description |
|
||||
|---|---|---|
|
||||
| `/api/voice/status` | GET | STT/TTS availability; `enabled` false if `VOICE_ENABLED=false` |
|
||||
| `/api/voice/voices` | GET | List available Kokoro voices |
|
||||
| `/api/voice/transcribe` | POST | multipart `audio` field → `{"transcript": "...", "duration_ms": 123}` |
|
||||
| `/api/voice/synthesise` | POST | `{"text", "voice", "speed"}` → WAV bytes |
|
||||
|
||||
---
|
||||
|
||||
## Modified Backend Files
|
||||
|
||||
### `src/fabledassistant/app.py`
|
||||
- Register `voice_bp` blueprint
|
||||
- In `startup()`: `asyncio.create_task(load_stt_model())` + `asyncio.create_task(load_tts_model())` when `VOICE_ENABLED`
|
||||
|
||||
### `src/fabledassistant/config.py`
|
||||
- Add 4 new env var attributes
|
||||
- Add validation in `validate()`
|
||||
|
||||
### `src/fabledassistant/services/llm.py`
|
||||
- Add `voice_mode: bool = False` and `voice_speech_style: str = "conversational"` to `build_context()`
|
||||
- When `voice_mode=True`, prepend: *"Respond naturally as if speaking aloud. No markdown, bullet points, headers, or code blocks. Complete sentences only."*
|
||||
- Append style modifier based on `voice_speech_style`
|
||||
|
||||
### `src/fabledassistant/services/generation_task.py`
|
||||
- Add `voice_mode: bool = False` to `run_generation()`
|
||||
- Read `voice_speech_style` from settings when voice_mode; pass both to `build_context()`
|
||||
|
||||
### `src/fabledassistant/routes/chat.py`
|
||||
- Allow `"voice"` in `conversation_type` whitelist
|
||||
|
||||
### `src/fabledassistant/services/chat.py`
|
||||
- Exclude `conversation_type == "voice"` from auto-cleanup retention
|
||||
|
||||
---
|
||||
|
||||
## New Frontend Files
|
||||
|
||||
### `frontend/src/composables/useVoiceRecorder.ts`
|
||||
Wraps `MediaRecorder`. Exports: `recording`, `error`, `isSupported`, `startRecording()`, `stopRecording() -> Promise<Blob>`.
|
||||
|
||||
### `frontend/src/composables/useVoiceAudio.ts`
|
||||
Wraps `AudioContext`. Exports: `playing`, `isSupported`, `play(blob)`, `stop()`.
|
||||
|
||||
### `frontend/src/components/VoiceOverlay.vue`
|
||||
Floating PTT button (fixed bottom-right). Creates/reuses a `"voice"` conversation. Full flow: record → transcribe → send → stream → synthesise → play. Space bar hotkey (from `App.vue`). Mounted globally in `App.vue`.
|
||||
|
||||
---
|
||||
|
||||
## Modified Frontend Files
|
||||
|
||||
### `frontend/src/api/client.ts`
|
||||
Add: `transcribeAudio(blob)`, `synthesiseSpeech(text, voice?, speed?)`, `getVoiceStatus()`, `getVoiceList()`
|
||||
|
||||
### `frontend/src/views/BriefingView.vue`
|
||||
- "Listen" button: reads latest assistant message aloud via TTS
|
||||
- Mic button in input bar: PTT → transcribe → auto-fill input → send
|
||||
- Auto-TTS on assistant response when in listen mode
|
||||
|
||||
### `frontend/src/views/SettingsView.vue`
|
||||
- New "Voice" tab: voice dropdown, speed slider, speech style radio
|
||||
- Loads from `/api/settings`, saves via `PUT /api/settings`
|
||||
|
||||
### `frontend/src/App.vue`
|
||||
- Mount `<VoiceOverlay />`
|
||||
- Space bar → `"voice:ptt-toggle"` custom event
|
||||
|
||||
---
|
||||
|
||||
## Audio Format
|
||||
|
||||
| Direction | Format | Rationale |
|
||||
|---|---|---|
|
||||
| Browser → Server | WebM/Opus | Native `MediaRecorder` output; no re-encoding |
|
||||
| Server → Browser | WAV (24kHz, 16-bit mono) | Kokoro native; no re-encoding; `decodeAudioData` compatible |
|
||||
|
||||
---
|
||||
|
||||
## Dependencies to Add (`pyproject.toml`)
|
||||
|
||||
```toml
|
||||
[project.optional-dependencies]
|
||||
voice = [
|
||||
"faster-whisper>=1.0",
|
||||
"kokoro>=0.9",
|
||||
"soundfile>=0.12",
|
||||
]
|
||||
```
|
||||
|
||||
Install unconditionally in Docker (activated by `VOICE_ENABLED` at runtime):
|
||||
```dockerfile
|
||||
RUN pip install faster-whisper kokoro soundfile
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Database Migration
|
||||
|
||||
No schema changes required. `conversation_type` is unconstrained TEXT. Voice settings use existing key-value `settings` table. Optional no-op migration `0034_voice_conversation_type.py` for audit trail.
|
||||
|
||||
---
|
||||
|
||||
## Implementation Phases
|
||||
|
||||
### Phase 1 — Backend services + routes
|
||||
1. Add env vars to `config.py`
|
||||
2. Create `services/stt.py` (faster-whisper)
|
||||
3. Create `services/tts.py` (Kokoro)
|
||||
4. Create `routes/voice.py` (4 endpoints)
|
||||
5. Wire model loading into `app.py` startup
|
||||
6. Add `voice_mode` to `build_context()` + `run_generation()`
|
||||
7. Allow `"voice"` conversation type in chat route + cleanup exclusion
|
||||
|
||||
### Phase 2 — BriefingView listen + voice follow-up
|
||||
1. Create `useVoiceRecorder.ts`
|
||||
2. Create `useVoiceAudio.ts`
|
||||
3. Add voice API functions to `client.ts`
|
||||
4. Add "Listen" button + mic button to `BriefingView.vue`
|
||||
|
||||
### Phase 3 — VoiceOverlay for general voice chat
|
||||
1. Create `VoiceOverlay.vue`
|
||||
2. Mount in `App.vue` + Space bar hotkey
|
||||
|
||||
### Phase 4 — Settings UI
|
||||
1. Add "Voice" tab to `SettingsView.vue`
|
||||
|
||||
---
|
||||
|
||||
## Kokoro Voice Reference
|
||||
|
||||
| ID | Character |
|
||||
|---|---|
|
||||
| `af_heart` | American female, warm (recommended default) |
|
||||
| `af_bella` | American female, expressive |
|
||||
| `af_nicole` | American female, breathy/intimate |
|
||||
| `af_sarah` | American female, clear |
|
||||
| `af_sky` | American female, bright |
|
||||
| `am_adam` | American male, neutral |
|
||||
| `am_michael` | American male, deeper |
|
||||
| `bf_emma` | British female |
|
||||
| `bf_isabella` | British female, formal |
|
||||
| `bm_george` | British male |
|
||||
| `bm_lewis` | British male, casual |
|
||||
@@ -298,7 +298,7 @@ async def fable_update_task(
|
||||
|
||||
|
||||
@mcp.tool()
|
||||
async def fable_add_task_log(task_id: int, body: str) -> dict:
|
||||
async def fable_add_task_log(task_id: int, content: str) -> dict:
|
||||
"""Append a timestamped progress log entry to a Fable task.
|
||||
|
||||
Use this to record work sessions, decisions, or status updates over time without
|
||||
@@ -306,7 +306,7 @@ async def fable_add_task_log(task_id: int, body: str) -> dict:
|
||||
chronologically in the task view.
|
||||
"""
|
||||
async with FableClient() as client:
|
||||
return await tasks.add_task_log(client, task_id=task_id, body=body)
|
||||
return await tasks.add_task_log(client, task_id=task_id, content=content)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
@@ -87,7 +87,7 @@ async def add_task_log(
|
||||
client: FableClient,
|
||||
*,
|
||||
task_id: int,
|
||||
body: str,
|
||||
content: str,
|
||||
) -> dict[str, Any]:
|
||||
"""Append a log entry to a task."""
|
||||
return await client.post(f"/api/tasks/{task_id}/logs", json={"body": body})
|
||||
return await client.post(f"/api/tasks/{task_id}/logs", json={"content": content})
|
||||
|
||||
@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
|
||||
|
||||
[project]
|
||||
name = "fable-mcp"
|
||||
version = "0.1.0"
|
||||
version = "0.2.0"
|
||||
description = "MCP server for Fabled Assistant"
|
||||
requires-python = ">=3.12"
|
||||
dependencies = [
|
||||
|
||||
@@ -46,9 +46,9 @@ async def test_update_task_patches(client):
|
||||
@pytest.mark.asyncio
|
||||
async def test_add_task_log_posts_to_logs_endpoint(client):
|
||||
from fable_mcp.tools.tasks import add_task_log
|
||||
client.post = AsyncMock(return_value={"id": 1, "body": "progress"})
|
||||
await add_task_log(client, task_id=9, body="progress")
|
||||
client.post = AsyncMock(return_value={"id": 1, "content": "progress"})
|
||||
await add_task_log(client, task_id=9, content="progress")
|
||||
client.post.assert_called_once()
|
||||
path = client.post.call_args[0][0]
|
||||
assert path == "/api/tasks/9/logs"
|
||||
assert client.post.call_args[1]["json"]["body"] == "progress"
|
||||
assert client.post.call_args[1]["json"]["content"] == "progress"
|
||||
|
||||
@@ -3,6 +3,7 @@ import { onMounted, onUnmounted, ref, watch } from "vue";
|
||||
import { useRouter } from "vue-router";
|
||||
import AppHeader from "@/components/AppHeader.vue";
|
||||
import ToastNotification from "@/components/ToastNotification.vue";
|
||||
import VoiceOverlay from "@/components/VoiceOverlay.vue";
|
||||
import { useTheme } from "@/composables/useTheme";
|
||||
import { useShortcuts } from "@/composables/useShortcuts";
|
||||
import { useAuthStore } from "@/stores/auth";
|
||||
@@ -47,9 +48,21 @@ function clearPrefix() {
|
||||
prefixTimeout = null;
|
||||
}
|
||||
|
||||
let spaceHeld = false;
|
||||
|
||||
function onGlobalKeydown(e: KeyboardEvent) {
|
||||
if (!authStore.isAuthenticated) return;
|
||||
|
||||
// Space — PTT start (only when not typing, no modifiers, no repeat)
|
||||
if (e.key === " " && !isInputActive() && !e.ctrlKey && !e.metaKey && !e.altKey && !e.repeat) {
|
||||
e.preventDefault();
|
||||
if (!spaceHeld) {
|
||||
spaceHeld = true;
|
||||
document.dispatchEvent(new CustomEvent("voice:ptt-toggle"));
|
||||
}
|
||||
return;
|
||||
}
|
||||
|
||||
// ? — toggle shortcuts overlay (only when not typing)
|
||||
if (e.key === "?" && !isInputActive() && !e.ctrlKey && !e.metaKey) {
|
||||
toggleShortcuts();
|
||||
@@ -122,8 +135,16 @@ function onGlobalKeydown(e: KeyboardEvent) {
|
||||
}
|
||||
}
|
||||
|
||||
function onGlobalKeyup(e: KeyboardEvent) {
|
||||
if (e.key === " " && spaceHeld) {
|
||||
spaceHeld = false;
|
||||
document.dispatchEvent(new CustomEvent("voice:ptt-toggle"));
|
||||
}
|
||||
}
|
||||
|
||||
onMounted(async () => {
|
||||
document.addEventListener("keydown", onGlobalKeydown);
|
||||
document.addEventListener("keyup", onGlobalKeyup);
|
||||
await authStore.checkAuth();
|
||||
if (authStore.isAuthenticated) {
|
||||
startAppServices();
|
||||
@@ -149,6 +170,7 @@ watch(
|
||||
|
||||
onUnmounted(() => {
|
||||
document.removeEventListener("keydown", onGlobalKeydown);
|
||||
document.removeEventListener("keyup", onGlobalKeyup);
|
||||
stopAppServices();
|
||||
});
|
||||
</script>
|
||||
@@ -164,6 +186,9 @@ onUnmounted(() => {
|
||||
<footer class="app-footer">v{{ appVersion }}</footer>
|
||||
</div>
|
||||
|
||||
<!-- Global voice PTT overlay -->
|
||||
<VoiceOverlay />
|
||||
|
||||
<!-- Keyboard shortcuts overlay -->
|
||||
<Transition name="shortcuts-fade">
|
||||
<div v-if="showShortcuts" class="shortcuts-overlay" @click.self="closeShortcuts">
|
||||
@@ -268,6 +293,10 @@ onUnmounted(() => {
|
||||
<kbd class="shortcut-key">Enter</kbd>
|
||||
<span class="shortcut-desc">New line</span>
|
||||
</div>
|
||||
<div class="shortcut-row">
|
||||
<kbd class="shortcut-key">Space</kbd>
|
||||
<span class="shortcut-desc">Hold to speak (voice, when enabled)</span>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
@@ -123,7 +123,6 @@ export interface NotificationEntry {
|
||||
export interface UserSearchResult {
|
||||
id: number
|
||||
username: string
|
||||
email: string | null
|
||||
}
|
||||
|
||||
// --- User search ---
|
||||
@@ -620,3 +619,51 @@ export function getNewsItems(params: GetNewsItemsParams = {}) {
|
||||
`/api/briefing/news?${p}`
|
||||
)
|
||||
}
|
||||
|
||||
// ─── Voice ────────────────────────────────────────────────────────────────────
|
||||
|
||||
export interface VoiceStatusResult {
|
||||
enabled: boolean
|
||||
stt: boolean
|
||||
tts: boolean
|
||||
stt_model?: string
|
||||
tts_backend?: string
|
||||
}
|
||||
|
||||
export interface VoiceEntry {
|
||||
id: string
|
||||
label: string
|
||||
}
|
||||
|
||||
export const getVoiceStatus = () => apiGet<VoiceStatusResult>('/api/voice/status')
|
||||
|
||||
export const getVoiceList = () =>
|
||||
apiGet<{ voices: VoiceEntry[] }>('/api/voice/voices').then(r => r.voices)
|
||||
|
||||
export async function transcribeAudio(blob: Blob): Promise<{ transcript: string; duration_ms: number }> {
|
||||
const form = new FormData()
|
||||
form.append('audio', blob, 'audio.webm')
|
||||
const res = await fetch('/api/voice/transcribe', { method: 'POST', body: form })
|
||||
if (!res.ok) {
|
||||
const err = await res.json().catch(() => ({ error: `HTTP ${res.status}` }))
|
||||
throw new ApiError(res.status, err)
|
||||
}
|
||||
return res.json()
|
||||
}
|
||||
|
||||
export async function synthesiseSpeech(
|
||||
text: string,
|
||||
voice?: string,
|
||||
speed?: number
|
||||
): Promise<Blob> {
|
||||
const res = await fetch('/api/voice/synthesise', {
|
||||
method: 'POST',
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
body: JSON.stringify({ text, voice: voice ?? 'af_heart', speed: speed ?? 1.0 }),
|
||||
})
|
||||
if (!res.ok) {
|
||||
const err = await res.json().catch(() => ({ error: `HTTP ${res.status}` }))
|
||||
throw new ApiError(res.status, err)
|
||||
}
|
||||
return res.blob()
|
||||
}
|
||||
|
||||
@@ -138,7 +138,6 @@ onMounted(async () => {
|
||||
<ul v-if="userResults.length" class="user-results">
|
||||
<li v-for="u in userResults" :key="u.id" @click="selectUser(u)" class="user-result-item">
|
||||
<span class="user-result-name">{{ u.username }}</span>
|
||||
<span class="user-result-email">{{ u.email }}</span>
|
||||
</li>
|
||||
</ul>
|
||||
</div>
|
||||
|
||||
@@ -0,0 +1,536 @@
|
||||
<script setup lang="ts">
|
||||
/**
|
||||
* VoiceOverlay — global floating push-to-talk button.
|
||||
*
|
||||
* Full flow: record → transcribe → send to voice conv → stream → TTS → play.
|
||||
* Manages its own "voice" conversation; does NOT touch the chat store so it
|
||||
* never disrupts an open chat session.
|
||||
*
|
||||
* Space bar toggles PTT when no input field is focused (wired from App.vue via
|
||||
* the "voice:ptt-toggle" custom event).
|
||||
*/
|
||||
import { ref, onMounted, onUnmounted, computed } from 'vue'
|
||||
import { useVoiceRecorder } from '@/composables/useVoiceRecorder'
|
||||
import { useVoiceAudio } from '@/composables/useVoiceAudio'
|
||||
import { apiPost, apiSSEStream, getVoiceStatus, transcribeAudio, synthesiseSpeech } from '@/api/client'
|
||||
|
||||
// ─── Voice service availability ──────────────────────────────────────────────
|
||||
const voiceEnabled = ref(false)
|
||||
|
||||
async function checkVoice() {
|
||||
try {
|
||||
const s = await getVoiceStatus()
|
||||
voiceEnabled.value = s.enabled && s.stt && s.tts
|
||||
} catch { /* feature absent */ }
|
||||
}
|
||||
|
||||
// ─── Conversation management ─────────────────────────────────────────────────
|
||||
const STORAGE_KEY = 'voice_overlay_conv_id'
|
||||
const convId = ref<number | null>(Number(localStorage.getItem(STORAGE_KEY)) || null)
|
||||
|
||||
interface VoiceMessage { role: 'user' | 'assistant'; content: string }
|
||||
const messages = ref<VoiceMessage[]>([])
|
||||
|
||||
async function ensureConversation(): Promise<number> {
|
||||
if (convId.value) {
|
||||
// Verify it still exists
|
||||
try {
|
||||
await fetch(`/api/chat/conversations/${convId.value}`)
|
||||
.then((r) => { if (!r.ok) throw new Error('gone') })
|
||||
return convId.value
|
||||
} catch {
|
||||
convId.value = null
|
||||
localStorage.removeItem(STORAGE_KEY)
|
||||
}
|
||||
}
|
||||
const conv = await apiPost<{ id: number }>('/api/chat/conversations', {
|
||||
title: 'Voice',
|
||||
conversation_type: 'voice',
|
||||
})
|
||||
convId.value = conv.id
|
||||
localStorage.setItem(STORAGE_KEY, String(conv.id))
|
||||
return conv.id
|
||||
}
|
||||
|
||||
// ─── State machine ────────────────────────────────────────────────────────────
|
||||
type Phase = 'idle' | 'recording' | 'transcribing' | 'generating' | 'speaking' | 'error'
|
||||
const phase = ref<Phase>('idle')
|
||||
const errorMsg = ref('')
|
||||
const streamContent = ref('')
|
||||
const open = ref(false)
|
||||
|
||||
const isBusy = computed(() => phase.value !== 'idle' && phase.value !== 'error')
|
||||
|
||||
// ─── Composables ─────────────────────────────────────────────────────────────
|
||||
const recorder = useVoiceRecorder()
|
||||
const audio = useVoiceAudio()
|
||||
|
||||
// ─── Core PTT flow ────────────────────────────────────────────────────────────
|
||||
async function startPtt() {
|
||||
if (!voiceEnabled.value || isBusy.value) return
|
||||
errorMsg.value = ''
|
||||
open.value = true
|
||||
await recorder.startRecording()
|
||||
if (recorder.error.value) {
|
||||
phase.value = 'error'
|
||||
errorMsg.value = recorder.error.value
|
||||
return
|
||||
}
|
||||
phase.value = 'recording'
|
||||
}
|
||||
|
||||
async function stopPtt() {
|
||||
if (phase.value !== 'recording') return
|
||||
|
||||
phase.value = 'transcribing'
|
||||
let blob: Blob
|
||||
try {
|
||||
blob = await recorder.stopRecording()
|
||||
} catch {
|
||||
phase.value = 'error'
|
||||
errorMsg.value = 'Recording failed'
|
||||
return
|
||||
}
|
||||
|
||||
let transcript: string
|
||||
try {
|
||||
const result = await transcribeAudio(blob)
|
||||
transcript = result.transcript.trim()
|
||||
} catch {
|
||||
phase.value = 'error'
|
||||
errorMsg.value = 'Transcription failed'
|
||||
return
|
||||
}
|
||||
|
||||
if (!transcript) {
|
||||
phase.value = 'idle'
|
||||
return
|
||||
}
|
||||
|
||||
messages.value.push({ role: 'user', content: transcript })
|
||||
scrollToBottom()
|
||||
|
||||
// Send to voice conversation
|
||||
phase.value = 'generating'
|
||||
streamContent.value = ''
|
||||
|
||||
let cid: number
|
||||
try {
|
||||
cid = await ensureConversation()
|
||||
} catch {
|
||||
phase.value = 'error'
|
||||
errorMsg.value = 'Could not create voice conversation'
|
||||
return
|
||||
}
|
||||
|
||||
let assistantMessageId: number
|
||||
try {
|
||||
const resp = await apiPost<{ assistant_message_id: number }>(
|
||||
`/api/chat/conversations/${cid}/messages`,
|
||||
{
|
||||
content: transcript,
|
||||
user_timezone: Intl.DateTimeFormat().resolvedOptions().timeZone,
|
||||
}
|
||||
)
|
||||
assistantMessageId = resp.assistant_message_id
|
||||
} catch {
|
||||
phase.value = 'error'
|
||||
errorMsg.value = 'Failed to send message'
|
||||
return
|
||||
}
|
||||
|
||||
// Stream the response
|
||||
await new Promise<void>((resolve) => {
|
||||
const handle = apiSSEStream(
|
||||
`/api/chat/conversations/${cid}/generation/stream`,
|
||||
(event) => {
|
||||
switch (event.event) {
|
||||
case 'chunk':
|
||||
streamContent.value += event.data.chunk as string
|
||||
break
|
||||
case 'done':
|
||||
handle.close()
|
||||
resolve()
|
||||
break
|
||||
case 'error':
|
||||
handle.close()
|
||||
resolve()
|
||||
break
|
||||
}
|
||||
}
|
||||
)
|
||||
// Safety timeout: 3 minutes
|
||||
setTimeout(() => { handle.close(); resolve() }, 180_000)
|
||||
})
|
||||
|
||||
const responseText = streamContent.value.trim()
|
||||
if (responseText) {
|
||||
messages.value.push({ role: 'assistant', content: responseText })
|
||||
streamContent.value = ''
|
||||
scrollToBottom()
|
||||
}
|
||||
|
||||
// Synthesise and play
|
||||
if (responseText) {
|
||||
phase.value = 'speaking'
|
||||
try {
|
||||
const wavBlob = await synthesiseSpeech(responseText)
|
||||
await audio.play(wavBlob)
|
||||
} catch {
|
||||
// TTS failure is non-critical; show response text
|
||||
}
|
||||
}
|
||||
|
||||
phase.value = 'idle'
|
||||
assistantMessageId // consumed; suppress lint
|
||||
}
|
||||
|
||||
function cancelAll() {
|
||||
recorder.stopRecording().catch(() => {})
|
||||
audio.stop()
|
||||
phase.value = 'idle'
|
||||
streamContent.value = ''
|
||||
errorMsg.value = ''
|
||||
}
|
||||
|
||||
// ─── Space bar PTT (event from App.vue) ──────────────────────────────────────
|
||||
function onPttToggle() {
|
||||
if (!voiceEnabled.value) return
|
||||
if (phase.value === 'recording') {
|
||||
stopPtt()
|
||||
} else if (phase.value === 'idle' || phase.value === 'error') {
|
||||
startPtt()
|
||||
}
|
||||
}
|
||||
|
||||
// ─── Scroll ───────────────────────────────────────────────────────────────────
|
||||
const transcriptEl = ref<HTMLElement | null>(null)
|
||||
function scrollToBottom() {
|
||||
setTimeout(() => {
|
||||
if (transcriptEl.value) {
|
||||
transcriptEl.value.scrollTop = transcriptEl.value.scrollHeight
|
||||
}
|
||||
}, 50)
|
||||
}
|
||||
|
||||
// ─── Lifecycle ────────────────────────────────────────────────────────────────
|
||||
onMounted(() => {
|
||||
checkVoice()
|
||||
document.addEventListener('voice:ptt-toggle', onPttToggle)
|
||||
})
|
||||
|
||||
onUnmounted(() => {
|
||||
document.removeEventListener('voice:ptt-toggle', onPttToggle)
|
||||
cancelAll()
|
||||
})
|
||||
</script>
|
||||
|
||||
<template>
|
||||
<Teleport to="body">
|
||||
<div v-if="voiceEnabled" class="voice-overlay">
|
||||
|
||||
<!-- Expanded transcript panel -->
|
||||
<Transition name="panel-slide">
|
||||
<div v-if="open && messages.length > 0" class="voice-panel">
|
||||
<div class="voice-panel-header">
|
||||
<span class="voice-panel-title">Voice</span>
|
||||
<button class="voice-panel-close" @click="open = false" aria-label="Close">×</button>
|
||||
</div>
|
||||
<div class="voice-transcript" ref="transcriptEl">
|
||||
<div
|
||||
v-for="(msg, i) in messages.slice(-10)"
|
||||
:key="i"
|
||||
:class="['voice-msg', `voice-msg--${msg.role}`]"
|
||||
>{{ msg.content }}</div>
|
||||
<!-- Live streaming text -->
|
||||
<div v-if="phase === 'generating' && streamContent" class="voice-msg voice-msg--assistant voice-msg--streaming">
|
||||
{{ streamContent }}<span class="voice-cursor">▌</span>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</Transition>
|
||||
|
||||
<!-- PTT button -->
|
||||
<div class="voice-btn-wrap">
|
||||
<!-- Status label -->
|
||||
<div v-if="phase !== 'idle'" class="voice-status-label">
|
||||
<span v-if="phase === 'recording'">Recording…</span>
|
||||
<span v-else-if="phase === 'transcribing'">Transcribing…</span>
|
||||
<span v-else-if="phase === 'generating'">Thinking…</span>
|
||||
<span v-else-if="phase === 'speaking'">Speaking…</span>
|
||||
<span v-else-if="phase === 'error'" class="voice-error-label">{{ errorMsg || 'Error' }}</span>
|
||||
</div>
|
||||
<div v-else-if="!open || !messages.length" class="voice-status-label voice-hint">
|
||||
Hold <kbd>Space</kbd> or tap
|
||||
</div>
|
||||
|
||||
<!-- Cancel button (shown while busy or speaking) -->
|
||||
<button
|
||||
v-if="isBusy || audio.playing.value"
|
||||
class="voice-cancel"
|
||||
@click="cancelAll"
|
||||
aria-label="Cancel"
|
||||
title="Cancel"
|
||||
>
|
||||
<svg width="13" height="13" viewBox="0 0 24 24" fill="currentColor">
|
||||
<path d="M19 6.41L17.59 5 12 10.59 6.41 5 5 6.41 10.59 12 5 17.59 6.41 19 12 13.41 17.59 19 19 17.59 13.41 12z"/>
|
||||
</svg>
|
||||
</button>
|
||||
|
||||
<!-- Main PTT button -->
|
||||
<button
|
||||
class="voice-ptt-btn"
|
||||
:class="{
|
||||
'voice-ptt--recording': phase === 'recording',
|
||||
'voice-ptt--busy': phase === 'generating' || phase === 'transcribing',
|
||||
'voice-ptt--speaking': phase === 'speaking' || audio.playing.value,
|
||||
'voice-ptt--error': phase === 'error',
|
||||
}"
|
||||
@mousedown.prevent="startPtt"
|
||||
@mouseup.prevent="stopPtt"
|
||||
@touchstart.prevent="startPtt"
|
||||
@touchend.prevent="stopPtt"
|
||||
@click.prevent="phase === 'error' ? (phase = 'idle') : undefined"
|
||||
:disabled="phase === 'transcribing' || phase === 'generating'"
|
||||
:aria-label="phase === 'recording' ? 'Release to send' : 'Hold to speak'"
|
||||
:title="phase === 'recording' ? 'Release to send' : 'Hold Space or tap to speak'"
|
||||
>
|
||||
<!-- Idle: mic icon -->
|
||||
<svg v-if="phase === 'idle' || phase === 'error'" width="22" height="22" viewBox="0 0 24 24" fill="currentColor">
|
||||
<path d="M12 14c1.66 0 3-1.34 3-3V5c0-1.66-1.34-3-3-3S9 3.34 9 5v6c0 1.66 1.34 3 3 3zm-1-9c0-.55.45-1 1-1s1 .45 1 1v6c0 .55-.45 1-1 1s-1-.45-1-1V5zm6 6c0 2.76-2.24 5-5 5s-5-2.24-5-5H5c0 3.53 2.61 6.43 6 6.92V21h2v-3.08c3.39-.49 6-3.39 6-6.92h-2z"/>
|
||||
</svg>
|
||||
<!-- Recording: waveform / stop icon -->
|
||||
<svg v-else-if="phase === 'recording'" width="22" height="22" viewBox="0 0 24 24" fill="currentColor">
|
||||
<path d="M6 19h4V5H6v14zm8-14v14h4V5h-4z"/>
|
||||
</svg>
|
||||
<!-- Busy: spinner dots -->
|
||||
<span v-else-if="phase === 'transcribing' || phase === 'generating'" class="voice-spinner">
|
||||
<span></span><span></span><span></span>
|
||||
</span>
|
||||
<!-- Speaking: sound waves -->
|
||||
<svg v-else-if="phase === 'speaking'" width="22" height="22" viewBox="0 0 24 24" fill="currentColor">
|
||||
<path d="M3 9v6h4l5 5V4L7 9H3zm13.5 3c0-1.77-1.02-3.29-2.5-4.03v8.05c1.48-.73 2.5-2.25 2.5-4.02zM14 3.23v2.06c2.89.86 5 3.54 5 6.71s-2.11 5.85-5 6.71v2.06c4.01-.91 7-4.49 7-8.77s-2.99-7.86-7-8.77z"/>
|
||||
</svg>
|
||||
</button>
|
||||
</div>
|
||||
|
||||
</div>
|
||||
</Teleport>
|
||||
</template>
|
||||
|
||||
<style scoped>
|
||||
.voice-overlay {
|
||||
position: fixed;
|
||||
bottom: 2rem;
|
||||
right: 1.5rem;
|
||||
z-index: 8000;
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
align-items: flex-end;
|
||||
gap: 0.5rem;
|
||||
pointer-events: none;
|
||||
}
|
||||
|
||||
/* ─── Panel ──────────────────────────────────────────────────────────────── */
|
||||
.voice-panel {
|
||||
pointer-events: all;
|
||||
width: min(320px, 90vw);
|
||||
background: var(--color-bg-card);
|
||||
border: 1px solid var(--color-border);
|
||||
border-radius: 14px;
|
||||
box-shadow: 0 8px 32px var(--color-shadow, rgba(0,0,0,0.22));
|
||||
overflow: hidden;
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
max-height: 340px;
|
||||
}
|
||||
|
||||
.voice-panel-header {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
justify-content: space-between;
|
||||
padding: 0.6rem 0.85rem 0.5rem;
|
||||
border-bottom: 1px solid var(--color-border);
|
||||
flex-shrink: 0;
|
||||
}
|
||||
.voice-panel-title {
|
||||
font-size: 0.75rem;
|
||||
font-weight: 700;
|
||||
text-transform: uppercase;
|
||||
letter-spacing: 0.07em;
|
||||
color: var(--color-primary);
|
||||
}
|
||||
.voice-panel-close {
|
||||
background: none;
|
||||
border: none;
|
||||
font-size: 1.3rem;
|
||||
line-height: 1;
|
||||
color: var(--color-text-muted);
|
||||
cursor: pointer;
|
||||
padding: 0 0.1rem;
|
||||
}
|
||||
.voice-panel-close:hover { color: var(--color-text); }
|
||||
|
||||
.voice-transcript {
|
||||
flex: 1;
|
||||
overflow-y: auto;
|
||||
padding: 0.65rem 0.85rem;
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
gap: 0.45rem;
|
||||
}
|
||||
|
||||
.voice-msg {
|
||||
font-size: 0.875rem;
|
||||
line-height: 1.45;
|
||||
padding: 0.4rem 0.65rem;
|
||||
border-radius: 10px;
|
||||
max-width: 88%;
|
||||
white-space: pre-wrap;
|
||||
word-break: break-word;
|
||||
}
|
||||
.voice-msg--user {
|
||||
align-self: flex-end;
|
||||
background: color-mix(in srgb, var(--color-primary) 12%, transparent);
|
||||
border: 1px solid color-mix(in srgb, var(--color-primary) 25%, transparent);
|
||||
color: var(--color-text);
|
||||
}
|
||||
.voice-msg--assistant {
|
||||
align-self: flex-start;
|
||||
background: var(--color-bg-secondary);
|
||||
border: 1px solid var(--color-border);
|
||||
color: var(--color-text);
|
||||
}
|
||||
.voice-msg--streaming { opacity: 0.85; }
|
||||
.voice-cursor {
|
||||
display: inline-block;
|
||||
animation: blink 0.9s step-end infinite;
|
||||
margin-left: 1px;
|
||||
color: var(--color-primary);
|
||||
}
|
||||
@keyframes blink { 0%, 100% { opacity: 1; } 50% { opacity: 0; } }
|
||||
|
||||
/* ─── Button cluster ─────────────────────────────────────────────────────── */
|
||||
.voice-btn-wrap {
|
||||
pointer-events: all;
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
align-items: flex-end;
|
||||
gap: 0.35rem;
|
||||
}
|
||||
|
||||
.voice-status-label {
|
||||
font-size: 0.72rem;
|
||||
color: var(--color-text-muted);
|
||||
background: var(--color-bg-card);
|
||||
border: 1px solid var(--color-border);
|
||||
border-radius: 6px;
|
||||
padding: 0.18rem 0.5rem;
|
||||
white-space: nowrap;
|
||||
box-shadow: 0 2px 6px var(--color-shadow, rgba(0,0,0,0.12));
|
||||
}
|
||||
.voice-hint { opacity: 0.7; }
|
||||
.voice-hint kbd {
|
||||
font-family: ui-monospace, monospace;
|
||||
font-size: 0.68rem;
|
||||
padding: 0.05rem 0.25rem;
|
||||
border: 1px solid var(--color-border);
|
||||
border-radius: 3px;
|
||||
background: var(--color-bg-secondary);
|
||||
}
|
||||
.voice-error-label { color: #ef4444; }
|
||||
|
||||
.voice-cancel {
|
||||
background: var(--color-bg-card);
|
||||
border: 1px solid var(--color-border);
|
||||
border-radius: 50%;
|
||||
width: 28px;
|
||||
height: 28px;
|
||||
display: flex;
|
||||
align-items: center;
|
||||
justify-content: center;
|
||||
cursor: pointer;
|
||||
color: var(--color-text-muted);
|
||||
box-shadow: 0 2px 6px var(--color-shadow, rgba(0,0,0,0.12));
|
||||
transition: all 0.15s;
|
||||
}
|
||||
.voice-cancel:hover { color: #ef4444; border-color: #ef4444; }
|
||||
|
||||
.voice-ptt-btn {
|
||||
width: 58px;
|
||||
height: 58px;
|
||||
border-radius: 50%;
|
||||
border: none;
|
||||
background: linear-gradient(135deg, #6366f1, #4f46e5);
|
||||
color: #fff;
|
||||
display: flex;
|
||||
align-items: center;
|
||||
justify-content: center;
|
||||
cursor: pointer;
|
||||
box-shadow: 0 4px 16px rgba(99, 102, 241, 0.45);
|
||||
transition: transform 0.12s, box-shadow 0.12s, background 0.2s;
|
||||
touch-action: none;
|
||||
user-select: none;
|
||||
flex-shrink: 0;
|
||||
}
|
||||
.voice-ptt-btn:hover:not(:disabled) {
|
||||
transform: scale(1.06);
|
||||
box-shadow: 0 6px 20px rgba(99, 102, 241, 0.55);
|
||||
}
|
||||
.voice-ptt-btn:disabled { opacity: 0.6; cursor: not-allowed; }
|
||||
|
||||
.voice-ptt--recording {
|
||||
background: linear-gradient(135deg, #ef4444, #dc2626) !important;
|
||||
box-shadow: 0 4px 16px rgba(239, 68, 68, 0.5) !important;
|
||||
animation: ptt-pulse 0.9s ease-in-out infinite;
|
||||
}
|
||||
.voice-ptt--busy {
|
||||
background: linear-gradient(135deg, #8b5cf6, #7c3aed) !important;
|
||||
box-shadow: 0 4px 16px rgba(139, 92, 246, 0.45) !important;
|
||||
}
|
||||
.voice-ptt--speaking {
|
||||
background: linear-gradient(135deg, #10b981, #059669) !important;
|
||||
box-shadow: 0 4px 16px rgba(16, 185, 129, 0.45) !important;
|
||||
animation: ptt-pulse 1.4s ease-in-out infinite;
|
||||
}
|
||||
.voice-ptt--error {
|
||||
background: linear-gradient(135deg, #6b7280, #4b5563) !important;
|
||||
box-shadow: none !important;
|
||||
}
|
||||
|
||||
@keyframes ptt-pulse {
|
||||
0%, 100% { transform: scale(1); }
|
||||
50% { transform: scale(1.08); }
|
||||
}
|
||||
|
||||
/* ─── Spinner dots ───────────────────────────────────────────────────────── */
|
||||
.voice-spinner {
|
||||
display: flex;
|
||||
gap: 4px;
|
||||
align-items: center;
|
||||
}
|
||||
.voice-spinner span {
|
||||
width: 5px;
|
||||
height: 5px;
|
||||
border-radius: 50%;
|
||||
background: #fff;
|
||||
animation: dot-bounce 1.2s ease-in-out infinite;
|
||||
}
|
||||
.voice-spinner span:nth-child(2) { animation-delay: 0.2s; }
|
||||
.voice-spinner span:nth-child(3) { animation-delay: 0.4s; }
|
||||
@keyframes dot-bounce {
|
||||
0%, 80%, 100% { transform: scale(0.7); opacity: 0.5; }
|
||||
40% { transform: scale(1); opacity: 1; }
|
||||
}
|
||||
|
||||
/* ─── Transition ─────────────────────────────────────────────────────────── */
|
||||
.panel-slide-enter-active,
|
||||
.panel-slide-leave-active {
|
||||
transition: opacity 0.2s ease, transform 0.2s ease;
|
||||
}
|
||||
.panel-slide-enter-from,
|
||||
.panel-slide-leave-to {
|
||||
opacity: 0;
|
||||
transform: translateY(8px);
|
||||
}
|
||||
</style>
|
||||
@@ -0,0 +1,60 @@
|
||||
import { ref, readonly } from 'vue'
|
||||
|
||||
/**
|
||||
* Audio playback composable wrapping the Web Audio API.
|
||||
*
|
||||
* Usage:
|
||||
* const { playing, isSupported, play, stop } = useVoiceAudio()
|
||||
* await play(wavBlob)
|
||||
*/
|
||||
export function useVoiceAudio() {
|
||||
const playing = ref(false)
|
||||
const isSupported = typeof AudioContext !== 'undefined' || typeof (window as unknown as Record<string, unknown>).webkitAudioContext !== 'undefined'
|
||||
|
||||
let audioCtx: AudioContext | null = null
|
||||
let currentSource: AudioBufferSourceNode | null = null
|
||||
|
||||
function _getCtx(): AudioContext {
|
||||
if (!audioCtx || audioCtx.state === 'closed') {
|
||||
const Ctx = window.AudioContext ?? (window as unknown as Record<string, typeof AudioContext>).webkitAudioContext
|
||||
audioCtx = new Ctx()
|
||||
}
|
||||
return audioCtx
|
||||
}
|
||||
|
||||
async function play(blob: Blob): Promise<void> {
|
||||
stop()
|
||||
const ctx = _getCtx()
|
||||
if (ctx.state === 'suspended') await ctx.resume()
|
||||
|
||||
const arrayBuffer = await blob.arrayBuffer()
|
||||
const audioBuffer = await ctx.decodeAudioData(arrayBuffer)
|
||||
|
||||
const source = ctx.createBufferSource()
|
||||
source.buffer = audioBuffer
|
||||
source.connect(ctx.destination)
|
||||
currentSource = source
|
||||
playing.value = true
|
||||
|
||||
source.onended = () => {
|
||||
playing.value = false
|
||||
currentSource = null
|
||||
}
|
||||
source.start(0)
|
||||
}
|
||||
|
||||
function stop(): void {
|
||||
if (currentSource) {
|
||||
try { currentSource.stop() } catch { /* already stopped */ }
|
||||
currentSource = null
|
||||
}
|
||||
playing.value = false
|
||||
}
|
||||
|
||||
return {
|
||||
playing: readonly(playing),
|
||||
isSupported,
|
||||
play,
|
||||
stop,
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,92 @@
|
||||
import { ref, readonly } from 'vue'
|
||||
|
||||
/**
|
||||
* Push-to-talk recorder wrapping the browser MediaRecorder API.
|
||||
*
|
||||
* Usage:
|
||||
* const { recording, error, isSupported, startRecording, stopRecording } = useVoiceRecorder()
|
||||
* await startRecording()
|
||||
* const blob = await stopRecording() // resolves with the recorded audio Blob
|
||||
*/
|
||||
export function useVoiceRecorder() {
|
||||
const recording = ref(false)
|
||||
const error = ref<string | null>(null)
|
||||
const isSupported = typeof MediaRecorder !== 'undefined' && !!navigator.mediaDevices?.getUserMedia
|
||||
|
||||
let mediaRecorder: MediaRecorder | null = null
|
||||
let chunks: Blob[] = []
|
||||
let stream: MediaStream | null = null
|
||||
let resolveStop: ((blob: Blob) => void) | null = null
|
||||
let rejectStop: ((err: Error) => void) | null = null
|
||||
|
||||
async function startRecording(): Promise<void> {
|
||||
error.value = null
|
||||
if (!isSupported) {
|
||||
error.value = 'Audio recording is not supported in this browser'
|
||||
return
|
||||
}
|
||||
if (recording.value) return
|
||||
|
||||
try {
|
||||
stream = await navigator.mediaDevices.getUserMedia({ audio: true })
|
||||
} catch (e) {
|
||||
error.value = 'Microphone access denied'
|
||||
return
|
||||
}
|
||||
|
||||
chunks = []
|
||||
const mimeType = MediaRecorder.isTypeSupported('audio/webm;codecs=opus')
|
||||
? 'audio/webm;codecs=opus'
|
||||
: MediaRecorder.isTypeSupported('audio/webm')
|
||||
? 'audio/webm'
|
||||
: ''
|
||||
|
||||
mediaRecorder = mimeType ? new MediaRecorder(stream, { mimeType }) : new MediaRecorder(stream)
|
||||
|
||||
mediaRecorder.ondataavailable = (e) => {
|
||||
if (e.data.size > 0) chunks.push(e.data)
|
||||
}
|
||||
|
||||
mediaRecorder.onstop = () => {
|
||||
const blob = new Blob(chunks, { type: mediaRecorder?.mimeType ?? 'audio/webm' })
|
||||
chunks = []
|
||||
stream?.getTracks().forEach((t) => t.stop())
|
||||
stream = null
|
||||
recording.value = false
|
||||
resolveStop?.(blob)
|
||||
resolveStop = null
|
||||
rejectStop = null
|
||||
}
|
||||
|
||||
mediaRecorder.onerror = () => {
|
||||
recording.value = false
|
||||
error.value = 'Recording error'
|
||||
rejectStop?.(new Error('MediaRecorder error'))
|
||||
resolveStop = null
|
||||
rejectStop = null
|
||||
}
|
||||
|
||||
mediaRecorder.start(100) // collect in 100ms chunks
|
||||
recording.value = true
|
||||
}
|
||||
|
||||
function stopRecording(): Promise<Blob> {
|
||||
return new Promise((resolve, reject) => {
|
||||
if (!mediaRecorder || !recording.value) {
|
||||
reject(new Error('Not recording'))
|
||||
return
|
||||
}
|
||||
resolveStop = resolve
|
||||
rejectStop = reject
|
||||
mediaRecorder.stop()
|
||||
})
|
||||
}
|
||||
|
||||
return {
|
||||
recording: readonly(recording),
|
||||
error: readonly(error),
|
||||
isSupported,
|
||||
startRecording,
|
||||
stopRecording,
|
||||
}
|
||||
}
|
||||
@@ -38,7 +38,7 @@ const headingRenderer = {
|
||||
marked.use({ renderer: headingRenderer });
|
||||
|
||||
const PURIFY_OPTS_FULL = {
|
||||
ADD_ATTR: ["data-tag", "data-title", "src", "alt"],
|
||||
ADD_ATTR: ["data-tag", "data-title"],
|
||||
FORCE_BODY: true,
|
||||
};
|
||||
|
||||
|
||||
@@ -1,6 +1,8 @@
|
||||
<script setup lang="ts">
|
||||
import { ref, computed, onMounted, watch, nextTick } from 'vue'
|
||||
import { useBackgroundRefresh } from '@/composables/useBackgroundRefresh'
|
||||
import { useVoiceRecorder } from '@/composables/useVoiceRecorder'
|
||||
import { useVoiceAudio } from '@/composables/useVoiceAudio'
|
||||
import { useChatStore } from '@/stores/chat'
|
||||
import ChatMessage from '@/components/ChatMessage.vue'
|
||||
import WeatherCard from '@/components/WeatherCard.vue'
|
||||
@@ -15,6 +17,9 @@ import {
|
||||
postRssReaction,
|
||||
deleteRssReaction,
|
||||
getNewsItems,
|
||||
getVoiceStatus,
|
||||
transcribeAudio,
|
||||
synthesiseSpeech,
|
||||
type BriefingConversation,
|
||||
type BriefingMessage,
|
||||
} from '@/api/client'
|
||||
@@ -224,6 +229,87 @@ function convLabel(c: BriefingConversation): string {
|
||||
return c.title || 'Briefing'
|
||||
}
|
||||
|
||||
// ─── Voice ────────────────────────────────────────────────────────────────────
|
||||
const voiceEnabled = ref(false)
|
||||
const listenMode = ref(false)
|
||||
const transcribing = ref(false)
|
||||
const synthesising = ref(false)
|
||||
|
||||
const recorder = useVoiceRecorder()
|
||||
const audio = useVoiceAudio()
|
||||
|
||||
// Check voice availability once on mount
|
||||
async function checkVoice() {
|
||||
try {
|
||||
const status = await getVoiceStatus()
|
||||
voiceEnabled.value = status.enabled && status.stt && status.tts
|
||||
} catch { /* voice feature absent */ }
|
||||
}
|
||||
|
||||
// Read the latest assistant message aloud
|
||||
async function listenToLatest() {
|
||||
const lastAssistant = [...messages.value].reverse().find((m) => m.role === 'assistant')
|
||||
if (!lastAssistant?.content) return
|
||||
await speakText(lastAssistant.content)
|
||||
}
|
||||
|
||||
async function speakText(text: string) {
|
||||
if (!voiceEnabled.value || synthesising.value) return
|
||||
// Strip markdown for cleaner TTS output
|
||||
const plain = text
|
||||
.replace(/```[\s\S]*?```/g, '')
|
||||
.replace(/`[^`]+`/g, (m) => m.slice(1, -1))
|
||||
.replace(/#{1,6}\s+/g, '')
|
||||
.replace(/\*\*([^*]+)\*\*/g, '$1')
|
||||
.replace(/\*([^*]+)\*/g, '$1')
|
||||
.replace(/\[([^\]]+)\]\([^)]+\)/g, '$1')
|
||||
.replace(/^\s*[-*+]\s+/gm, '')
|
||||
.replace(/\n{2,}/g, ' ')
|
||||
.trim()
|
||||
if (!plain) return
|
||||
synthesising.value = true
|
||||
try {
|
||||
const blob = await synthesiseSpeech(plain)
|
||||
await audio.play(blob)
|
||||
} catch {
|
||||
// TTS failure is non-critical
|
||||
} finally {
|
||||
synthesising.value = false
|
||||
}
|
||||
}
|
||||
|
||||
// PTT: hold to record, release to transcribe and send
|
||||
async function startPtt() {
|
||||
if (!voiceEnabled.value || recorder.recording.value) return
|
||||
await recorder.startRecording()
|
||||
}
|
||||
|
||||
async function stopPtt() {
|
||||
if (!recorder.recording.value) return
|
||||
transcribing.value = true
|
||||
try {
|
||||
const blob = await recorder.stopRecording()
|
||||
const { transcript } = await transcribeAudio(blob)
|
||||
if (transcript.trim()) {
|
||||
input.value = transcript.trim()
|
||||
await send()
|
||||
}
|
||||
} catch {
|
||||
// transcription failure — leave input empty
|
||||
} finally {
|
||||
transcribing.value = false
|
||||
}
|
||||
}
|
||||
|
||||
// Auto-TTS when listen mode is on and streaming ends
|
||||
watch(() => chatStore.streaming, async (streaming) => {
|
||||
if (!streaming && listenMode.value && voiceEnabled.value) {
|
||||
// Small delay to let messages update after stream
|
||||
await new Promise((r) => setTimeout(r, 200))
|
||||
await listenToLatest()
|
||||
}
|
||||
})
|
||||
|
||||
// Convert BriefingMessage to Message for ChatMessage component
|
||||
function toMsg(m: BriefingMessage): Message {
|
||||
return {
|
||||
@@ -261,6 +347,7 @@ useBackgroundRefresh(
|
||||
onMounted(async () => {
|
||||
await checkSetup()
|
||||
if (!showWizard.value) await loadAll()
|
||||
checkVoice()
|
||||
})
|
||||
</script>
|
||||
|
||||
@@ -281,6 +368,29 @@ onMounted(async () => {
|
||||
<select v-if="conversations.length" v-model="selectedConvId" class="briefing-conv-select">
|
||||
<option v-for="c in conversations" :key="c.id" :value="c.id">{{ convLabel(c) }}</option>
|
||||
</select>
|
||||
<!-- Listen button: reads latest assistant message; toggles auto-TTS mode -->
|
||||
<button
|
||||
v-if="voiceEnabled"
|
||||
class="btn-voice-header"
|
||||
:class="{ 'btn-voice-active': listenMode, 'btn-voice-busy': synthesising || audio.playing.value }"
|
||||
@click="listenMode ? (listenMode = false) : (listenMode = true, listenToLatest())"
|
||||
:disabled="synthesising"
|
||||
:title="listenMode ? 'Stop auto-read' : 'Listen to briefing'"
|
||||
>
|
||||
<svg v-if="!synthesising && !audio.playing.value" width="15" height="15" viewBox="0 0 24 24" fill="currentColor">
|
||||
<path d="M3 9v6h4l5 5V4L7 9H3zm13.5 3c0-1.77-1.02-3.29-2.5-4.03v8.05c1.48-.73 2.5-2.25 2.5-4.02zM14 3.23v2.06c2.89.86 5 3.54 5 6.71s-2.11 5.85-5 6.71v2.06c4.01-.91 7-4.49 7-8.77s-2.99-7.86-7-8.77z"/>
|
||||
</svg>
|
||||
<svg v-else width="15" height="15" viewBox="0 0 24 24" fill="currentColor">
|
||||
<path d="M12 3v10.55c-.59-.34-1.27-.55-2-.55-2.21 0-4 1.79-4 4s1.79 4 4 4 4-1.79 4-4V7h4V3h-6z"/>
|
||||
</svg>
|
||||
{{ listenMode ? 'Listening' : 'Listen' }}
|
||||
</button>
|
||||
<button
|
||||
v-if="voiceEnabled && (synthesising || audio.playing.value)"
|
||||
class="btn-trigger"
|
||||
@click="audio.stop()"
|
||||
title="Stop playback"
|
||||
>Stop</button>
|
||||
<button
|
||||
class="btn-trigger"
|
||||
@click="triggerNow"
|
||||
@@ -343,10 +453,31 @@ onMounted(async () => {
|
||||
<textarea
|
||||
v-model="input"
|
||||
class="briefing-input"
|
||||
placeholder="Reply to your briefing…"
|
||||
:placeholder="transcribing ? 'Transcribing…' : recorder.recording.value ? 'Recording…' : 'Reply to your briefing…'"
|
||||
rows="1"
|
||||
@keydown="onKeydown"
|
||||
:disabled="transcribing || recorder.recording.value"
|
||||
></textarea>
|
||||
<!-- Mic PTT button (hold to record) -->
|
||||
<button
|
||||
v-if="voiceEnabled && recorder.isSupported"
|
||||
class="btn-mic"
|
||||
:class="{ 'btn-mic-active': recorder.recording.value, 'btn-mic-busy': transcribing }"
|
||||
@mousedown.prevent="startPtt"
|
||||
@mouseup.prevent="stopPtt"
|
||||
@touchstart.prevent="startPtt"
|
||||
@touchend.prevent="stopPtt"
|
||||
:disabled="transcribing || chatStore.streaming || sending"
|
||||
:title="recorder.recording.value ? 'Release to send' : 'Hold to speak'"
|
||||
aria-label="Push to talk"
|
||||
>
|
||||
<svg v-if="!transcribing" width="16" height="16" viewBox="0 0 24 24" fill="currentColor">
|
||||
<path d="M12 14c1.66 0 3-1.34 3-3V5c0-1.66-1.34-3-3-3S9 3.34 9 5v6c0 1.66 1.34 3 3 3zm-1-9c0-.55.45-1 1-1s1 .45 1 1v6c0 .55-.45 1-1 1s-1-.45-1-1V5zm6 6c0 2.76-2.24 5-5 5s-5-2.24-5-5H5c0 3.53 2.61 6.43 6 6.92V21h2v-3.08c3.39-.49 6-3.39 6-6.92h-2z"/>
|
||||
</svg>
|
||||
<svg v-else width="16" height="16" viewBox="0 0 24 24" fill="currentColor">
|
||||
<path d="M12 4V1L8 5l4 4V6c3.31 0 6 2.69 6 6 0 1.01-.25 1.97-.7 2.8l1.46 1.46C19.54 15.03 20 13.57 20 12c0-4.42-3.58-8-8-8zm0 14c-3.31 0-6-2.69-6-6 0-1.01.25-1.97.7-2.8L5.24 7.74C4.46 8.97 4 10.43 4 12c0 4.42 3.58 8 8 8v3l4-4-4-4v3z"/>
|
||||
</svg>
|
||||
</button>
|
||||
<button
|
||||
class="btn-send"
|
||||
@click="send"
|
||||
@@ -706,6 +837,74 @@ a.news-title:hover { text-decoration: underline; color: var(--color-primary); }
|
||||
background: color-mix(in srgb, var(--color-primary) 12%, transparent);
|
||||
}
|
||||
|
||||
/* ─── Voice buttons ──────────────────────────────────────────────────────── */
|
||||
|
||||
.btn-voice-header {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
gap: 0.35rem;
|
||||
padding: 0.35rem 0.75rem;
|
||||
border: 1px solid var(--color-border);
|
||||
border-radius: 6px;
|
||||
background: var(--color-bg-card);
|
||||
color: var(--color-text-muted);
|
||||
font-size: 0.8rem;
|
||||
cursor: pointer;
|
||||
white-space: nowrap;
|
||||
transition: all 0.15s;
|
||||
font-family: inherit;
|
||||
}
|
||||
.btn-voice-header:hover:not(:disabled) {
|
||||
border-color: var(--color-primary);
|
||||
color: var(--color-primary);
|
||||
}
|
||||
.btn-voice-header:disabled { opacity: 0.5; cursor: not-allowed; }
|
||||
.btn-voice-active {
|
||||
border-color: var(--color-primary) !important;
|
||||
color: var(--color-primary) !important;
|
||||
background: color-mix(in srgb, var(--color-primary) 10%, transparent) !important;
|
||||
}
|
||||
.btn-voice-busy {
|
||||
border-color: var(--color-primary);
|
||||
color: var(--color-primary);
|
||||
animation: pulse-border 1.2s ease-in-out infinite;
|
||||
}
|
||||
@keyframes pulse-border {
|
||||
0%, 100% { opacity: 1; }
|
||||
50% { opacity: 0.55; }
|
||||
}
|
||||
|
||||
.btn-mic {
|
||||
padding: 0.55rem 0.65rem;
|
||||
background: var(--color-bg-secondary);
|
||||
border: 1px solid var(--color-border);
|
||||
border-radius: 10px;
|
||||
color: var(--color-text-muted);
|
||||
cursor: pointer;
|
||||
display: flex;
|
||||
align-items: center;
|
||||
justify-content: center;
|
||||
flex-shrink: 0;
|
||||
transition: all 0.15s;
|
||||
touch-action: none;
|
||||
user-select: none;
|
||||
}
|
||||
.btn-mic:hover:not(:disabled) {
|
||||
border-color: var(--color-primary);
|
||||
color: var(--color-primary);
|
||||
}
|
||||
.btn-mic:disabled { opacity: 0.4; cursor: not-allowed; }
|
||||
.btn-mic-active {
|
||||
background: color-mix(in srgb, #ef4444 15%, transparent) !important;
|
||||
border-color: #ef4444 !important;
|
||||
color: #ef4444 !important;
|
||||
animation: pulse-border 0.8s ease-in-out infinite;
|
||||
}
|
||||
.btn-mic-busy {
|
||||
border-color: var(--color-primary);
|
||||
color: var(--color-primary);
|
||||
}
|
||||
|
||||
/* ─── Responsive ─────────────────────────────────────────────────────────── */
|
||||
|
||||
@media (max-width: 900px) {
|
||||
|
||||
@@ -3,7 +3,7 @@ import { ref, watch, onMounted } from "vue";
|
||||
import { useSettingsStore } from "@/stores/settings";
|
||||
import { useAuthStore } from "@/stores/auth";
|
||||
import { useToastStore } from "@/stores/toast";
|
||||
import { apiGet, apiPost, apiPut, apiDelete, listGroups, createGroup, deleteGroup, listGroupMembers, addGroupMember, removeGroupMember, searchUsers, getBriefingConfig, saveBriefingConfig, getBriefingFeeds, createBriefingFeed, deleteBriefingFeed, refreshBriefingFeeds, geocodeAddress, getFableMcpInfo, listApiKeys, createApiKey as apiCreateApiKey, revokeApiKey as apiRevokeApiKey, type ApiKeyEntry, type GroupEntry, type GroupMember, type UserSearchResult, type BriefingConfig, type BriefingFeed } from "@/api/client";
|
||||
import { apiGet, apiPost, apiPut, apiDelete, listGroups, createGroup, deleteGroup, listGroupMembers, addGroupMember, removeGroupMember, searchUsers, getBriefingConfig, saveBriefingConfig, getBriefingFeeds, createBriefingFeed, deleteBriefingFeed, refreshBriefingFeeds, geocodeAddress, getFableMcpInfo, listApiKeys, createApiKey as apiCreateApiKey, revokeApiKey as apiRevokeApiKey, getVoiceStatus, getVoiceList, type ApiKeyEntry, type GroupEntry, type GroupMember, type UserSearchResult, type BriefingConfig, type BriefingFeed, type VoiceStatusResult, type VoiceEntry } from "@/api/client";
|
||||
import { usePushStore } from "@/stores/push";
|
||||
import type { User } from "@/types/auth";
|
||||
import PaginationBar from "@/components/PaginationBar.vue";
|
||||
@@ -40,7 +40,7 @@ const appVersion = ref('dev');
|
||||
const restoreFileInput = ref<HTMLInputElement | null>(null);
|
||||
|
||||
// Migrate stored "admin" → "config"; unknown tabs fall back to "general"
|
||||
const VALID_TABS = new Set(["general", "account", "notifications", "integrations", "data", "briefing", "apikeys", "config", "users", "logs", "groups"]);
|
||||
const VALID_TABS = new Set(["general", "account", "notifications", "integrations", "data", "briefing", "voice", "apikeys", "config", "users", "logs", "groups"]);
|
||||
const _stored = localStorage.getItem("settings_tab") ?? "general";
|
||||
const activeTab = ref(VALID_TABS.has(_stored) ? (_stored === "admin" ? "config" : _stored) : "general");
|
||||
|
||||
@@ -51,6 +51,7 @@ function _loadTabContent(tab: string) {
|
||||
else if (tab === "groups") loadGroupsPanel();
|
||||
}
|
||||
if (tab === "briefing") loadBriefingTab();
|
||||
if (tab === "voice") loadVoiceTab();
|
||||
if (tab === "apikeys") { fetchApiKeys(); loadMcpInfo(); }
|
||||
}
|
||||
|
||||
@@ -444,6 +445,56 @@ const baseUrl = ref("");
|
||||
const savingBaseUrl = ref(false);
|
||||
const baseUrlSaved = ref(false);
|
||||
|
||||
// Voice config (admin only)
|
||||
const adminVoiceEnabled = ref(false);
|
||||
const adminVoiceSttModel = ref("base.en");
|
||||
const savingAdminVoice = ref(false);
|
||||
const adminVoiceSaved = ref(false);
|
||||
const voiceLoadingModels = ref(false);
|
||||
|
||||
async function saveAdminVoice() {
|
||||
savingAdminVoice.value = true;
|
||||
adminVoiceSaved.value = false;
|
||||
try {
|
||||
await apiPut("/api/admin/voice", {
|
||||
voice_enabled: adminVoiceEnabled.value,
|
||||
voice_stt_model: adminVoiceSttModel.value,
|
||||
});
|
||||
adminVoiceSaved.value = true;
|
||||
setTimeout(() => { adminVoiceSaved.value = false; }, 2000);
|
||||
voiceTabLoaded.value = false;
|
||||
if (adminVoiceEnabled.value) {
|
||||
await reloadVoiceModels();
|
||||
}
|
||||
} finally {
|
||||
savingAdminVoice.value = false;
|
||||
}
|
||||
}
|
||||
|
||||
async function reloadVoiceModels() {
|
||||
voiceLoadingModels.value = true;
|
||||
try {
|
||||
await apiPost("/api/admin/voice/reload", {});
|
||||
// Poll /api/voice/status until both STT and TTS are ready (max 3 min)
|
||||
const deadline = Date.now() + 3 * 60 * 1000;
|
||||
while (Date.now() < deadline) {
|
||||
await new Promise((r) => setTimeout(r, 2500));
|
||||
try {
|
||||
const status = await apiGet<{ enabled: boolean; stt: boolean; tts: boolean }>("/api/voice/status");
|
||||
if (status.stt && status.tts) {
|
||||
voiceStatus.value = status;
|
||||
voiceTabLoaded.value = true;
|
||||
break;
|
||||
}
|
||||
} catch { /* keep polling */ }
|
||||
}
|
||||
} catch {
|
||||
toastStore.show("Failed to trigger model reload", "error");
|
||||
} finally {
|
||||
voiceLoadingModels.value = false;
|
||||
}
|
||||
}
|
||||
|
||||
// Search test (SearXNG)
|
||||
const searxngConfigured = ref(false);
|
||||
const searxngUrl = ref("");
|
||||
@@ -452,6 +503,49 @@ const searchResults = ref<{ url: string; title: string; snippet: string }[]>([])
|
||||
const searchLoading = ref(false);
|
||||
const searchError = ref("");
|
||||
|
||||
// Voice settings
|
||||
const voiceStatus = ref<VoiceStatusResult | null>(null);
|
||||
const voiceStatusLoading = ref(false);
|
||||
const availableVoices = ref<VoiceEntry[]>([]);
|
||||
const voiceTtsVoice = ref("af_heart");
|
||||
const voiceTtsSpeed = ref(1.0);
|
||||
const voiceSpeechStyle = ref("conversational");
|
||||
const savingVoice = ref(false);
|
||||
const voiceSaved = ref(false);
|
||||
const voiceTabLoaded = ref(false);
|
||||
|
||||
async function loadVoiceTab() {
|
||||
if (voiceTabLoaded.value) return;
|
||||
voiceTabLoaded.value = true;
|
||||
voiceStatusLoading.value = true;
|
||||
try {
|
||||
voiceStatus.value = await getVoiceStatus();
|
||||
if (voiceStatus.value.tts) {
|
||||
availableVoices.value = await getVoiceList();
|
||||
}
|
||||
} catch {
|
||||
// Voice feature may be disabled
|
||||
} finally {
|
||||
voiceStatusLoading.value = false;
|
||||
}
|
||||
}
|
||||
|
||||
async function saveVoiceSettings() {
|
||||
savingVoice.value = true;
|
||||
voiceSaved.value = false;
|
||||
try {
|
||||
await apiPut("/api/settings", {
|
||||
voice_tts_voice: voiceTtsVoice.value,
|
||||
voice_tts_speed: String(voiceTtsSpeed.value),
|
||||
voice_speech_style: voiceSpeechStyle.value,
|
||||
});
|
||||
voiceSaved.value = true;
|
||||
setTimeout(() => { voiceSaved.value = false; }, 2000);
|
||||
} finally {
|
||||
savingVoice.value = false;
|
||||
}
|
||||
}
|
||||
|
||||
onMounted(async () => {
|
||||
try {
|
||||
const v = await apiGet<{ version: string }>('/api/version')
|
||||
@@ -485,6 +579,11 @@ onMounted(async () => {
|
||||
notifySecurityAlerts.value = allSettings.notify_security_alerts !== "false";
|
||||
}
|
||||
|
||||
// Load voice settings
|
||||
if (allSettings.voice_tts_voice) voiceTtsVoice.value = allSettings.voice_tts_voice;
|
||||
if (allSettings.voice_tts_speed) voiceTtsSpeed.value = parseFloat(allSettings.voice_tts_speed);
|
||||
if (allSettings.voice_speech_style) voiceSpeechStyle.value = allSettings.voice_speech_style;
|
||||
|
||||
// Load CalDAV settings
|
||||
try {
|
||||
const caldavConfig = await apiGet<Record<string, string>>("/api/settings/caldav");
|
||||
@@ -516,6 +615,13 @@ onMounted(async () => {
|
||||
} catch {
|
||||
// base URL not configured yet
|
||||
}
|
||||
try {
|
||||
const vc = await apiGet<Record<string, string>>("/api/admin/voice");
|
||||
adminVoiceEnabled.value = vc.voice_enabled === "true";
|
||||
adminVoiceSttModel.value = vc.voice_stt_model ?? "base.en";
|
||||
} catch {
|
||||
// voice not configured yet
|
||||
}
|
||||
}
|
||||
_loadTabContent(activeTab.value);
|
||||
});
|
||||
@@ -1134,7 +1240,7 @@ function formatUserDate(iso: string): string {
|
||||
<div class="sidebar-group">
|
||||
<div class="sidebar-group-label">User</div>
|
||||
<button
|
||||
v-for="tab in ['general', 'account', 'notifications', 'integrations', 'data', 'briefing', 'apikeys']"
|
||||
v-for="tab in ['general', 'account', 'notifications', 'integrations', 'data', 'briefing', 'voice', 'apikeys']"
|
||||
:key="tab"
|
||||
:class="['sidebar-item', { active: activeTab === tab }]"
|
||||
@click="activeTab = tab"
|
||||
@@ -1810,6 +1916,104 @@ function formatUserDate(iso: string): string {
|
||||
|
||||
</div>
|
||||
|
||||
<!-- ── Voice ── -->
|
||||
<div v-show="activeTab === 'voice'" class="settings-grid">
|
||||
|
||||
<section class="settings-section full-width">
|
||||
<h2>Voice</h2>
|
||||
<p class="section-desc">
|
||||
Configure text-to-speech and speech-to-text for voice conversations.
|
||||
Requires <code>VOICE_ENABLED=true</code> on the server.
|
||||
</p>
|
||||
|
||||
<!-- Status indicator -->
|
||||
<div v-if="voiceStatusLoading" class="field-hint">Loading voice status…</div>
|
||||
<div v-else-if="voiceStatus === null" class="field-hint">Voice status unavailable.</div>
|
||||
<div v-else class="voice-status-row">
|
||||
<span :class="['status-badge', voiceStatus.enabled ? 'status-on' : 'status-off']">
|
||||
{{ voiceStatus.enabled ? 'Enabled' : 'Disabled' }}
|
||||
</span>
|
||||
<span v-if="voiceStatus.enabled" class="status-badge" :class="voiceStatus.stt ? 'status-on' : 'status-off'">
|
||||
STT {{ voiceStatus.stt ? 'ready' : 'loading…' }}
|
||||
</span>
|
||||
<span v-if="voiceStatus.enabled" class="status-badge" :class="voiceStatus.tts ? 'status-on' : 'status-off'">
|
||||
TTS {{ voiceStatus.tts ? 'ready' : 'loading…' }}
|
||||
</span>
|
||||
<span v-if="voiceStatus.stt_model" class="field-hint" style="margin-left:0.5rem;">
|
||||
Model: {{ voiceStatus.stt_model }}
|
||||
</span>
|
||||
</div>
|
||||
|
||||
<div v-if="!voiceStatus?.enabled" class="field-hint" style="margin-top:0.75rem;">
|
||||
Voice is disabled. An administrator can enable it under
|
||||
<strong>Admin → Config → Voice</strong>.
|
||||
</div>
|
||||
</section>
|
||||
|
||||
<section v-if="voiceStatus?.enabled" class="settings-section full-width">
|
||||
<h2>Speech Style</h2>
|
||||
<p class="section-desc">Controls how the assistant phrases spoken responses.</p>
|
||||
|
||||
<div class="field-group">
|
||||
<label class="field-label">Style</label>
|
||||
<div class="radio-group">
|
||||
<label v-for="opt in [
|
||||
{ value: 'conversational', label: 'Conversational', hint: 'Warm and natural, like speaking to a friend' },
|
||||
{ value: 'concise', label: 'Concise', hint: 'Short and to the point' },
|
||||
{ value: 'detailed', label: 'Detailed', hint: 'Thorough explanations spoken aloud' },
|
||||
]" :key="opt.value" class="radio-option">
|
||||
<input type="radio" v-model="voiceSpeechStyle" :value="opt.value" />
|
||||
<span>
|
||||
<strong>{{ opt.label }}</strong>
|
||||
<span class="field-hint">{{ opt.hint }}</span>
|
||||
</span>
|
||||
</label>
|
||||
</div>
|
||||
</div>
|
||||
</section>
|
||||
|
||||
<section v-if="voiceStatus?.enabled" class="settings-section full-width">
|
||||
<h2>Voice & Speed</h2>
|
||||
<p class="section-desc">Choose the TTS voice and playback speed.</p>
|
||||
|
||||
<div class="field-group">
|
||||
<label class="field-label" for="voice-select">Voice</label>
|
||||
<select id="voice-select" class="input" v-model="voiceTtsVoice" :disabled="availableVoices.length === 0">
|
||||
<option v-if="availableVoices.length === 0" value="af_heart">af_heart (default)</option>
|
||||
<option v-for="v in availableVoices" :key="v.id" :value="v.id">{{ v.label }}</option>
|
||||
</select>
|
||||
<p class="field-hint">Voice character and accent. Applies to all voice conversations.</p>
|
||||
</div>
|
||||
|
||||
<div class="field-group">
|
||||
<label class="field-label" for="voice-speed">
|
||||
Speed: {{ voiceTtsSpeed.toFixed(1) }}×
|
||||
</label>
|
||||
<input
|
||||
id="voice-speed"
|
||||
type="range"
|
||||
min="0.7"
|
||||
max="1.3"
|
||||
step="0.05"
|
||||
v-model.number="voiceTtsSpeed"
|
||||
class="range-input"
|
||||
/>
|
||||
<div class="range-labels">
|
||||
<span>0.7× (slow)</span>
|
||||
<span>1.0× (normal)</span>
|
||||
<span>1.3× (fast)</span>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div class="form-actions">
|
||||
<button class="btn-save" @click="saveVoiceSettings" :disabled="savingVoice">
|
||||
{{ voiceSaved ? 'Saved ✓' : savingVoice ? 'Saving…' : 'Save Voice Settings' }}
|
||||
</button>
|
||||
</div>
|
||||
</section>
|
||||
|
||||
</div>
|
||||
|
||||
<!-- ── API Keys ── -->
|
||||
<div v-show="activeTab === 'apikeys'" class="settings-grid">
|
||||
<section class="settings-section full-width">
|
||||
@@ -1998,6 +2202,50 @@ FABLE_API_KEY=<your-api-key></pre>
|
||||
</div>
|
||||
</section>
|
||||
|
||||
<section class="settings-section full-width">
|
||||
<h2>Voice (Speech-to-Speech)</h2>
|
||||
<p class="section-desc">
|
||||
Enable self-hosted voice using faster-whisper (STT) and Kokoro (TTS).
|
||||
Models load at startup — a server restart is required after changing these settings.
|
||||
Install dependencies first: <code>pip install faster-whisper kokoro soundfile</code>
|
||||
</p>
|
||||
<div class="checkbox-field">
|
||||
<label>
|
||||
<input type="checkbox" v-model="adminVoiceEnabled" />
|
||||
Enable Voice
|
||||
</label>
|
||||
<p class="field-hint">Shows the PTT button and voice settings to all users.</p>
|
||||
</div>
|
||||
<div v-if="adminVoiceEnabled" class="field" style="max-width:280px; margin-top:0.75rem;">
|
||||
<label for="stt-model">STT Model</label>
|
||||
<select id="stt-model" v-model="adminVoiceSttModel" class="input">
|
||||
<option value="tiny.en">tiny.en — fastest, lowest accuracy (~75 MB)</option>
|
||||
<option value="base.en">base.en — balanced (recommended, ~145 MB)</option>
|
||||
<option value="small.en">small.en — better accuracy (~465 MB)</option>
|
||||
<option value="medium.en">medium.en — best accuracy (~1.5 GB)</option>
|
||||
</select>
|
||||
<p class="field-hint">Larger models are more accurate but use more RAM and take longer to load.</p>
|
||||
</div>
|
||||
<div class="actions" style="margin-top:0.85rem;">
|
||||
<button class="btn-save" @click="saveAdminVoice" :disabled="savingAdminVoice || voiceLoadingModels">
|
||||
{{ savingAdminVoice ? 'Saving…' : adminVoiceSaved ? 'Saved ✓' : 'Save' }}
|
||||
</button>
|
||||
<button
|
||||
v-if="!voiceLoadingModels && adminVoiceEnabled"
|
||||
class="btn-secondary"
|
||||
@click="reloadVoiceModels"
|
||||
:disabled="voiceLoadingModels"
|
||||
title="Reload models without restarting the server"
|
||||
>Reload models</button>
|
||||
<span v-if="voiceLoadingModels" class="field-hint">
|
||||
Loading models… this may take a minute.
|
||||
<span class="voice-admin-spinner">
|
||||
<span></span><span></span><span></span>
|
||||
</span>
|
||||
</span>
|
||||
</div>
|
||||
</section>
|
||||
|
||||
</div>
|
||||
|
||||
<!-- ── Users ── -->
|
||||
@@ -2263,7 +2511,6 @@ FABLE_API_KEY=<your-api-key></pre>
|
||||
<ul v-if="groupMemberResults.length" class="member-results">
|
||||
<li v-for="u in groupMemberResults" :key="u.id" class="member-result-item" @click="addMemberToGroup(g.id, u)">
|
||||
<span class="member-result-name">{{ u.username }}</span>
|
||||
<span class="member-result-email">{{ u.email }}</span>
|
||||
</li>
|
||||
</ul>
|
||||
</div>
|
||||
@@ -3529,4 +3776,82 @@ FABLE_API_KEY=<your-api-key></pre>
|
||||
word-break: break-all;
|
||||
overflow-x: auto;
|
||||
}
|
||||
|
||||
/* Voice tab */
|
||||
.voice-status-row {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
gap: 0.5rem;
|
||||
flex-wrap: wrap;
|
||||
margin-top: 0.5rem;
|
||||
}
|
||||
.status-badge {
|
||||
font-size: 0.75rem;
|
||||
font-weight: 600;
|
||||
padding: 0.15rem 0.55rem;
|
||||
border-radius: 999px;
|
||||
}
|
||||
.status-on {
|
||||
background: color-mix(in srgb, var(--color-success, #22c55e) 15%, transparent);
|
||||
color: var(--color-success, #22c55e);
|
||||
border: 1px solid color-mix(in srgb, var(--color-success, #22c55e) 40%, transparent);
|
||||
}
|
||||
.status-off {
|
||||
background: color-mix(in srgb, var(--color-text-muted) 10%, transparent);
|
||||
color: var(--color-text-muted);
|
||||
border: 1px solid var(--color-border);
|
||||
}
|
||||
.radio-group {
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
gap: 0.65rem;
|
||||
margin-top: 0.35rem;
|
||||
}
|
||||
.radio-option {
|
||||
display: flex;
|
||||
align-items: flex-start;
|
||||
gap: 0.55rem;
|
||||
cursor: pointer;
|
||||
}
|
||||
.radio-option input[type="radio"] { margin-top: 0.2rem; flex-shrink: 0; }
|
||||
.radio-option span { display: flex; flex-direction: column; gap: 0.15rem; }
|
||||
.radio-option strong { font-size: 0.875rem; }
|
||||
.range-input {
|
||||
width: 100%;
|
||||
max-width: 360px;
|
||||
accent-color: var(--color-primary);
|
||||
margin: 0.35rem 0 0.2rem;
|
||||
}
|
||||
.range-labels {
|
||||
display: flex;
|
||||
justify-content: space-between;
|
||||
max-width: 360px;
|
||||
font-size: 0.75rem;
|
||||
color: var(--color-text-muted);
|
||||
}
|
||||
.form-actions {
|
||||
margin-top: 1rem;
|
||||
}
|
||||
|
||||
.voice-admin-spinner {
|
||||
display: inline-flex;
|
||||
gap: 3px;
|
||||
align-items: center;
|
||||
margin-left: 0.4rem;
|
||||
vertical-align: middle;
|
||||
}
|
||||
.voice-admin-spinner span {
|
||||
display: inline-block;
|
||||
width: 4px;
|
||||
height: 4px;
|
||||
border-radius: 50%;
|
||||
background: var(--color-text-muted);
|
||||
animation: va-dot-bounce 1.2s ease-in-out infinite;
|
||||
}
|
||||
.voice-admin-spinner span:nth-child(2) { animation-delay: 0.2s; }
|
||||
.voice-admin-spinner span:nth-child(3) { animation-delay: 0.4s; }
|
||||
@keyframes va-dot-bounce {
|
||||
0%, 80%, 100% { transform: scale(0.6); opacity: 0.4; }
|
||||
40% { transform: scale(1); opacity: 1; }
|
||||
}
|
||||
</style>
|
||||
|
||||
@@ -29,6 +29,11 @@ dev = [
|
||||
"pytest-asyncio>=0.23",
|
||||
"ruff>=0.6",
|
||||
]
|
||||
voice = [
|
||||
"faster-whisper>=1.0",
|
||||
"kokoro>=0.9",
|
||||
"soundfile>=0.12",
|
||||
]
|
||||
|
||||
[tool.setuptools.packages.find]
|
||||
where = ["src"]
|
||||
|
||||
@@ -31,6 +31,7 @@ from fabledassistant.routes.users import users_bp
|
||||
from fabledassistant.routes.api_keys import api_keys_bp
|
||||
from fabledassistant.routes.events import events_bp
|
||||
from fabledassistant.routes.search import search_bp
|
||||
from fabledassistant.routes.voice import voice_bp
|
||||
|
||||
STATIC_DIR = Path(__file__).parent / "static"
|
||||
logger = logging.getLogger(__name__)
|
||||
@@ -62,6 +63,13 @@ def create_app() -> Quart:
|
||||
"Set SECRET_KEY or SECRET_KEY_FILE for production use."
|
||||
)
|
||||
|
||||
if not Config.TRUST_PROXY_HEADERS:
|
||||
logger.warning(
|
||||
"TRUST_PROXY_HEADERS is not set. If this instance is behind a reverse proxy "
|
||||
"(nginx, Caddy, Traefik) set TRUST_PROXY_HEADERS=true so rate limiting uses "
|
||||
"real client IPs rather than the proxy IP."
|
||||
)
|
||||
|
||||
app.register_blueprint(admin_bp)
|
||||
app.register_blueprint(api)
|
||||
app.register_blueprint(auth_bp)
|
||||
@@ -85,6 +93,7 @@ def create_app() -> Quart:
|
||||
app.register_blueprint(api_keys_bp)
|
||||
app.register_blueprint(events_bp)
|
||||
app.register_blueprint(search_bp)
|
||||
app.register_blueprint(voice_bp)
|
||||
|
||||
@app.before_request
|
||||
async def before_request():
|
||||
@@ -257,6 +266,12 @@ def create_app() -> Quart:
|
||||
from fabledassistant.services.briefing_scheduler import start_briefing_scheduler
|
||||
await start_briefing_scheduler(asyncio.get_running_loop())
|
||||
|
||||
# Voice model loading (enabled via Admin → Config in the UI, or VOICE_ENABLED env var)
|
||||
from fabledassistant.services.stt import load_stt_model
|
||||
from fabledassistant.services.tts import load_tts_model
|
||||
asyncio.create_task(load_stt_model())
|
||||
asyncio.create_task(load_tts_model())
|
||||
|
||||
@app.after_serving
|
||||
async def shutdown():
|
||||
from fabledassistant.services.briefing_scheduler import stop_briefing_scheduler
|
||||
|
||||
@@ -66,6 +66,12 @@ class Config:
|
||||
VAPID_PUBLIC_KEY: str = os.environ.get("VAPID_PUBLIC_KEY", "")
|
||||
VAPID_CLAIMS_SUB: str = os.environ.get("VAPID_CLAIMS_SUB", "mailto:admin@fabledassistant.local")
|
||||
|
||||
# Voice (Speech-to-Speech) feature
|
||||
VOICE_ENABLED: bool = os.environ.get("VOICE_ENABLED", "").lower() in ("1", "true", "yes")
|
||||
STT_BACKEND: str = os.environ.get("STT_BACKEND", "faster-whisper")
|
||||
STT_MODEL: str = os.environ.get("STT_MODEL", "base.en")
|
||||
TTS_BACKEND: str = os.environ.get("TTS_BACKEND", "kokoro")
|
||||
|
||||
@classmethod
|
||||
def oidc_enabled(cls) -> bool:
|
||||
return bool(cls.OIDC_ISSUER and cls.OIDC_CLIENT_ID and cls.OIDC_CLIENT_SECRET)
|
||||
@@ -88,5 +94,16 @@ class Config:
|
||||
errors.append(f"SMTP_PORT={cls.SMTP_PORT} must be between 1 and 65535")
|
||||
if cls.oidc_enabled() and not cls.BASE_URL.startswith(("http://", "https://")):
|
||||
errors.append(f"BASE_URL='{cls.BASE_URL}' must start with http:// or https:// when OIDC is enabled")
|
||||
if cls.SECRET_KEY == "dev-secret-change-me" and cls.SECURE_COOKIES:
|
||||
errors.append(
|
||||
"SECRET_KEY is set to the insecure default but SECURE_COOKIES=true indicates "
|
||||
"a production deployment. Set SECRET_KEY or SECRET_KEY_FILE before starting."
|
||||
)
|
||||
_valid_stt_models = {"tiny.en", "base.en", "small.en", "medium.en"}
|
||||
if cls.VOICE_ENABLED and cls.STT_MODEL not in _valid_stt_models:
|
||||
errors.append(
|
||||
f"STT_MODEL='{cls.STT_MODEL}' is not supported. "
|
||||
f"Valid values: {', '.join(sorted(_valid_stt_models))}"
|
||||
)
|
||||
if errors:
|
||||
raise ValueError("Configuration errors:\n" + "\n".join(f" - {e}" for e in errors))
|
||||
|
||||
@@ -1,3 +1,12 @@
|
||||
"""In-process sliding-window rate limiter.
|
||||
|
||||
IMPORTANT — deployment note:
|
||||
Rate limit counters are stored in memory and are lost on process restart.
|
||||
When deployed behind a reverse proxy (nginx, Caddy, Traefik) you MUST set
|
||||
TRUST_PROXY_HEADERS=true so that the real client IP is used as the bucket key
|
||||
rather than the proxy's IP (which would cause all users to share one bucket).
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import time
|
||||
from collections import defaultdict
|
||||
|
||||
@@ -19,6 +19,7 @@ from fabledassistant.services.backup import (
|
||||
restore_full_backup,
|
||||
)
|
||||
from fabledassistant.services.email import SMTP_SETTING_KEYS, get_base_url, get_smtp_config, is_smtp_configured, send_test_email
|
||||
from fabledassistant.services.voice_config import get_voice_config
|
||||
from fabledassistant.services.logging import get_logs, get_log_stats, log_audit
|
||||
from fabledassistant.services.notifications import send_invitation_email
|
||||
from fabledassistant.services.settings import set_setting, set_settings_batch
|
||||
@@ -200,6 +201,44 @@ async def update_base_url():
|
||||
return jsonify({"status": "ok"})
|
||||
|
||||
|
||||
@admin_bp.route("/voice", methods=["GET"])
|
||||
@admin_required
|
||||
async def get_voice_config_route():
|
||||
config = await get_voice_config()
|
||||
return jsonify(config)
|
||||
|
||||
|
||||
@admin_bp.route("/voice", methods=["PUT"])
|
||||
@admin_required
|
||||
async def update_voice_config():
|
||||
data = await request.get_json()
|
||||
uid = get_current_user_id()
|
||||
valid_models = {"tiny.en", "base.en", "small.en", "medium.en"}
|
||||
settings: dict[str, str] = {}
|
||||
if "voice_enabled" in data:
|
||||
settings["voice_enabled"] = "true" if data["voice_enabled"] else "false"
|
||||
if "voice_stt_model" in data:
|
||||
model = str(data["voice_stt_model"])
|
||||
if model not in valid_models:
|
||||
return jsonify({"error": f"Invalid STT model. Choose from: {', '.join(sorted(valid_models))}"}), 400
|
||||
settings["voice_stt_model"] = model
|
||||
if settings:
|
||||
await set_settings_batch(uid, settings)
|
||||
await log_audit("voice_config", user_id=uid, username=g.user.username, ip_address=request.remote_addr, details=settings)
|
||||
return jsonify({"status": "ok"})
|
||||
|
||||
|
||||
@admin_bp.route("/voice/reload", methods=["POST"])
|
||||
@admin_required
|
||||
async def reload_voice_models():
|
||||
"""Reload STT and TTS models in the background without a server restart."""
|
||||
from fabledassistant.services.stt import reload_stt_model
|
||||
from fabledassistant.services.tts import reload_tts_model
|
||||
asyncio.create_task(reload_stt_model())
|
||||
asyncio.create_task(reload_tts_model())
|
||||
return jsonify({"status": "loading"})
|
||||
|
||||
|
||||
@admin_bp.route("/invitations", methods=["POST"])
|
||||
@admin_required
|
||||
async def create_invite():
|
||||
|
||||
@@ -359,7 +359,10 @@ async def oauth_callback():
|
||||
return redirect("/login?error=oauth")
|
||||
|
||||
sub = claims.get("sub", "")
|
||||
email = claims.get("email", "")
|
||||
# Only trust the email claim for account linking if the provider has verified it.
|
||||
# An unverified email could be used to hijack an existing local account.
|
||||
email_verified = claims.get("email_verified", False)
|
||||
email = claims.get("email", "") if email_verified else ""
|
||||
preferred_username = claims.get("preferred_username", "")
|
||||
|
||||
if not sub:
|
||||
|
||||
@@ -71,6 +71,12 @@ async def add_feed():
|
||||
url = (data.get("url") or "").strip()
|
||||
if not url:
|
||||
return jsonify({"error": "url required"}), 400
|
||||
scheme = url.split("://")[0].lower() if "://" in url else ""
|
||||
if scheme not in ("http", "https"):
|
||||
return jsonify({"error": "Feed URL must use http or https"}), 400
|
||||
from fabledassistant.services.llm import _is_private_url
|
||||
if _is_private_url(url):
|
||||
return jsonify({"error": "Feed URL must not point to an internal address"}), 400
|
||||
category = data.get("category") or None
|
||||
|
||||
async with async_session() as session:
|
||||
|
||||
@@ -5,7 +5,7 @@ import logging
|
||||
import httpx
|
||||
from quart import Blueprint, Response, jsonify, request
|
||||
|
||||
from fabledassistant.auth import login_required, get_current_user_id
|
||||
from fabledassistant.auth import admin_required, login_required, get_current_user_id
|
||||
from fabledassistant.routes.utils import not_found, parse_pagination
|
||||
from fabledassistant.config import Config
|
||||
from fabledassistant.services.chat import (
|
||||
@@ -76,7 +76,7 @@ async def create_conversation_route():
|
||||
model = data.get("model", Config.OLLAMA_MODEL)
|
||||
conversation_type = data.get("conversation_type", "chat")
|
||||
# Only allow known types to prevent accidental misuse
|
||||
if conversation_type not in ("chat", "mcp"):
|
||||
if conversation_type not in ("chat", "mcp", "voice"):
|
||||
conversation_type = "chat"
|
||||
conv = await create_conversation(uid, title=title, model=model, conversation_type=conversation_type)
|
||||
return jsonify(conv.to_dict()), 201
|
||||
@@ -187,6 +187,7 @@ async def send_message_route(conv_id: int):
|
||||
rag_project_id=effective_rag_project_id,
|
||||
workspace_project_id=workspace_project_id,
|
||||
user_timezone=user_timezone,
|
||||
voice_mode=(conv.conversation_type == "voice"),
|
||||
))
|
||||
|
||||
return jsonify({
|
||||
@@ -439,7 +440,7 @@ async def list_models_route():
|
||||
|
||||
|
||||
@chat_bp.route("/models/pull", methods=["POST"])
|
||||
@login_required
|
||||
@admin_required
|
||||
async def pull_model_route():
|
||||
"""Pull a model from Ollama, streaming progress via SSE."""
|
||||
data = await request.get_json()
|
||||
@@ -478,7 +479,7 @@ async def pull_model_route():
|
||||
|
||||
|
||||
@chat_bp.route("/models/delete", methods=["POST"])
|
||||
@login_required
|
||||
@admin_required
|
||||
async def delete_model_route():
|
||||
"""Delete a model from Ollama."""
|
||||
data = await request.get_json()
|
||||
|
||||
@@ -1,14 +1,10 @@
|
||||
"""Serve locally-cached images.
|
||||
|
||||
No authentication required — image IDs are opaque and unguessable (based
|
||||
on the SHA-256 of the original URL). Images are served with a 1-day
|
||||
Cache-Control header so the browser doesn't re-request on every page load.
|
||||
"""
|
||||
"""Serve locally-cached images."""
|
||||
|
||||
import logging
|
||||
|
||||
from quart import Blueprint, jsonify, send_file
|
||||
|
||||
from fabledassistant.auth import login_required
|
||||
from fabledassistant.services.images import get_image_path, get_image_record
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
@@ -17,6 +13,7 @@ images_bp = Blueprint("images", __name__, url_prefix="/api/images")
|
||||
|
||||
|
||||
@images_bp.route("/<int:image_id>", methods=["GET"])
|
||||
@login_required
|
||||
async def serve_image(image_id: int):
|
||||
"""Serve a locally-cached image by its DB ID."""
|
||||
record = await get_image_record(image_id)
|
||||
|
||||
@@ -50,12 +50,16 @@ async def create_milestone_route(project_id: int):
|
||||
data = await request.get_json()
|
||||
if not data.get("title"):
|
||||
return jsonify({"error": "title is required"}), 400
|
||||
status = data.get("status", "active")
|
||||
if status not in ("active", "done"):
|
||||
return jsonify({"error": "status must be 'active' or 'done'"}), 400
|
||||
milestone = await create_milestone(
|
||||
uid,
|
||||
project_id,
|
||||
title=data["title"],
|
||||
description=data.get("description"),
|
||||
order_index=data.get("order_index", 0),
|
||||
status=status,
|
||||
)
|
||||
return jsonify(await _milestone_dict(milestone)), 201
|
||||
|
||||
|
||||
@@ -38,12 +38,16 @@ async def create_project_route():
|
||||
data = await request.get_json()
|
||||
if not data.get("title"):
|
||||
return jsonify({"error": "title is required"}), 400
|
||||
status = data.get("status", "active")
|
||||
if status not in ("active", "archived"):
|
||||
return jsonify({"error": "status must be 'active' or 'archived'"}), 400
|
||||
project = await create_project(
|
||||
uid,
|
||||
title=data["title"],
|
||||
description=data.get("description", ""),
|
||||
goal=data.get("goal", ""),
|
||||
color=data.get("color"),
|
||||
status=status,
|
||||
)
|
||||
return jsonify(project.to_dict()), 201
|
||||
|
||||
|
||||
@@ -3,7 +3,7 @@ from quart import Blueprint, jsonify, request
|
||||
from fabledassistant.auth import login_required, get_current_user_id
|
||||
from fabledassistant.config import Config
|
||||
from fabledassistant.services.caldav import CALDAV_SETTING_KEYS, get_caldav_config, test_connection
|
||||
from fabledassistant.services.llm import get_installed_models
|
||||
from fabledassistant.services.llm import get_installed_models, _is_private_url
|
||||
from fabledassistant.services.settings import delete_setting, get_all_settings, set_settings_batch
|
||||
|
||||
settings_bp = Blueprint("settings", __name__, url_prefix="/api/settings")
|
||||
@@ -77,6 +77,16 @@ async def update_caldav():
|
||||
uid = get_current_user_id()
|
||||
data = await request.get_json()
|
||||
|
||||
# Validate CalDAV URL before saving — block internal/private addresses
|
||||
if "caldav_url" in data:
|
||||
url = str(data.get("caldav_url") or "").strip()
|
||||
if url:
|
||||
parsed_scheme = url.split("://")[0].lower() if "://" in url else ""
|
||||
if parsed_scheme not in ("http", "https"):
|
||||
return jsonify({"error": "CalDAV URL must use http or https"}), 400
|
||||
if _is_private_url(url):
|
||||
return jsonify({"error": "CalDAV URL must not point to an internal or private address"}), 400
|
||||
|
||||
settings_to_save = {}
|
||||
for key in CALDAV_SETTING_KEYS:
|
||||
if key in data:
|
||||
|
||||
@@ -24,6 +24,6 @@ async def search_users():
|
||||
).limit(10)
|
||||
)).scalars().all()
|
||||
return jsonify({"users": [
|
||||
{"id": u.id, "username": u.username, "email": u.email}
|
||||
{"id": u.id, "username": u.username}
|
||||
for u in users
|
||||
]})
|
||||
|
||||
@@ -0,0 +1,136 @@
|
||||
"""Voice (Speech-to-Speech) routes at /api/voice."""
|
||||
import logging
|
||||
import time
|
||||
|
||||
from quart import Blueprint, jsonify, request
|
||||
|
||||
from fabledassistant.auth import login_required
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
voice_bp = Blueprint("voice", __name__, url_prefix="/api/voice")
|
||||
|
||||
|
||||
@voice_bp.route("/status", methods=["GET"])
|
||||
@login_required
|
||||
async def voice_status():
|
||||
"""Return availability of STT and TTS services."""
|
||||
from fabledassistant.services.voice_config import get_voice_config
|
||||
from fabledassistant.services.stt import stt_available
|
||||
from fabledassistant.services.tts import tts_available
|
||||
|
||||
config = await get_voice_config()
|
||||
enabled = config.get("voice_enabled", "false").lower() in ("1", "true", "yes")
|
||||
|
||||
if not enabled:
|
||||
return jsonify({"enabled": False, "stt": False, "tts": False})
|
||||
|
||||
return jsonify({
|
||||
"enabled": True,
|
||||
"stt": stt_available(),
|
||||
"tts": tts_available(),
|
||||
"stt_model": config.get("voice_stt_model", "base.en"),
|
||||
"tts_backend": "kokoro",
|
||||
})
|
||||
|
||||
|
||||
@voice_bp.route("/voices", methods=["GET"])
|
||||
@login_required
|
||||
async def list_voices():
|
||||
"""Return available Kokoro voice IDs and labels."""
|
||||
from fabledassistant.services.voice_config import is_voice_enabled
|
||||
if not await is_voice_enabled():
|
||||
return jsonify({"error": "Voice feature is disabled"}), 503
|
||||
|
||||
from fabledassistant.services.tts import list_voices, tts_available
|
||||
|
||||
if not tts_available():
|
||||
return jsonify({"error": "TTS not available"}), 503
|
||||
|
||||
return jsonify({"voices": list_voices()})
|
||||
|
||||
|
||||
@voice_bp.route("/transcribe", methods=["POST"])
|
||||
@login_required
|
||||
async def transcribe_audio():
|
||||
"""Accept a multipart audio file and return the transcript.
|
||||
|
||||
Request: multipart/form-data with field 'audio' (WebM/Opus blob)
|
||||
Response: {"transcript": "...", "duration_ms": 123}
|
||||
"""
|
||||
from fabledassistant.services.voice_config import is_voice_enabled
|
||||
if not await is_voice_enabled():
|
||||
return jsonify({"error": "Voice feature is disabled"}), 503
|
||||
|
||||
from fabledassistant.services.stt import stt_available, transcribe
|
||||
|
||||
if not stt_available():
|
||||
return jsonify({"error": "STT not available — model may still be loading"}), 503
|
||||
|
||||
files = await request.files
|
||||
audio_file = files.get("audio")
|
||||
if audio_file is None:
|
||||
return jsonify({"error": "No audio file provided"}), 400
|
||||
|
||||
audio_bytes = audio_file.read()
|
||||
if not audio_bytes:
|
||||
return jsonify({"error": "Empty audio file"}), 400
|
||||
|
||||
if len(audio_bytes) > 25 * 1024 * 1024: # 25 MB hard cap
|
||||
return jsonify({"error": "Audio file too large (max 25 MB)"}), 413
|
||||
|
||||
mime_type = audio_file.content_type or "audio/webm"
|
||||
|
||||
t0 = time.monotonic()
|
||||
try:
|
||||
transcript = await transcribe(audio_bytes, mime_type)
|
||||
except Exception:
|
||||
logger.exception("STT transcription failed")
|
||||
return jsonify({"error": "Transcription failed"}), 500
|
||||
|
||||
duration_ms = round((time.monotonic() - t0) * 1000)
|
||||
return jsonify({"transcript": transcript, "duration_ms": duration_ms})
|
||||
|
||||
|
||||
@voice_bp.route("/synthesise", methods=["POST"])
|
||||
@login_required
|
||||
async def synthesise_speech():
|
||||
"""Convert text to speech and return WAV bytes.
|
||||
|
||||
Request body: {"text": "...", "voice": "af_heart", "speed": 1.0}
|
||||
Response: audio/wav bytes
|
||||
"""
|
||||
from fabledassistant.services.voice_config import is_voice_enabled
|
||||
if not await is_voice_enabled():
|
||||
return jsonify({"error": "Voice feature is disabled"}), 503
|
||||
|
||||
from fabledassistant.services.tts import synthesise, tts_available
|
||||
|
||||
if not tts_available():
|
||||
return jsonify({"error": "TTS not available — model may still be loading"}), 503
|
||||
|
||||
data = await request.get_json()
|
||||
if not data:
|
||||
return jsonify({"error": "JSON body required"}), 400
|
||||
|
||||
text = str(data.get("text", "")).strip()
|
||||
if not text:
|
||||
return jsonify({"error": "text is required"}), 400
|
||||
|
||||
if len(text) > 8000:
|
||||
return jsonify({"error": "text too long (max 8000 characters)"}), 400
|
||||
|
||||
voice = str(data.get("voice", "af_heart"))
|
||||
try:
|
||||
speed = float(data.get("speed", 1.0))
|
||||
except (TypeError, ValueError):
|
||||
speed = 1.0
|
||||
|
||||
try:
|
||||
wav_bytes = await synthesise(text, voice=voice, speed=speed)
|
||||
except Exception:
|
||||
logger.exception("TTS synthesis failed")
|
||||
return jsonify({"error": "Synthesis failed"}), 500
|
||||
|
||||
from quart import Response
|
||||
return Response(wav_bytes, mimetype="audio/wav")
|
||||
@@ -131,7 +131,8 @@ async def cleanup_old_conversations(user_id: int, days: int) -> int:
|
||||
.where(
|
||||
Conversation.user_id == user_id,
|
||||
Conversation.updated_at < cutoff,
|
||||
Conversation.conversation_type != "mcp", # preserve MCP audit trail
|
||||
Conversation.conversation_type != "mcp", # preserve MCP audit trail
|
||||
Conversation.conversation_type != "voice", # voice convs managed separately
|
||||
)
|
||||
.returning(Conversation.id)
|
||||
)
|
||||
|
||||
@@ -152,6 +152,7 @@ async def run_generation(
|
||||
rag_project_id: int | None = None,
|
||||
workspace_project_id: int | None = None,
|
||||
user_timezone: str | None = None,
|
||||
voice_mode: bool = False,
|
||||
) -> None:
|
||||
"""Stream LLM response into buffer with periodic DB flushes."""
|
||||
MAX_TOOL_ROUNDS = 5
|
||||
@@ -177,6 +178,12 @@ async def run_generation(
|
||||
# Phase 3: Build context and wait for model in parallel.
|
||||
model_load_task = asyncio.create_task(wait_for_model_loaded(model, timeout=180.0))
|
||||
|
||||
# Fetch voice_speech_style from user settings when voice_mode is active.
|
||||
voice_speech_style = "conversational"
|
||||
if voice_mode:
|
||||
from fabledassistant.services.settings import get_setting
|
||||
voice_speech_style = await get_setting(user_id, "voice_speech_style", "conversational")
|
||||
|
||||
context_task = asyncio.create_task(build_context(
|
||||
user_id, history_to_use, context_note_id, user_content,
|
||||
history_summary=history_summary,
|
||||
@@ -186,6 +193,8 @@ async def run_generation(
|
||||
workspace_project_id=workspace_project_id,
|
||||
user_timezone=user_timezone,
|
||||
conv_id=conv_id,
|
||||
voice_mode=voice_mode,
|
||||
voice_speech_style=voice_speech_style,
|
||||
))
|
||||
|
||||
messages, context_meta = await context_task
|
||||
|
||||
@@ -7,7 +7,9 @@ twice shares one file and one DB row.
|
||||
"""
|
||||
|
||||
import hashlib
|
||||
import ipaddress
|
||||
import logging
|
||||
import socket
|
||||
from pathlib import Path
|
||||
from urllib.parse import urlparse
|
||||
|
||||
@@ -21,6 +23,27 @@ from fabledassistant.models.image_cache import ImageCache
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
def _is_safe_image_url(url: str) -> bool:
|
||||
"""Return True only if the URL is a public http/https address (SSRF guard)."""
|
||||
try:
|
||||
parsed = urlparse(url)
|
||||
if parsed.scheme not in ("http", "https"):
|
||||
return False
|
||||
host = parsed.hostname
|
||||
if not host:
|
||||
return False
|
||||
if host.lower() in ("localhost", "::1"):
|
||||
return False
|
||||
addr_info = socket.getaddrinfo(host, None, proto=socket.IPPROTO_TCP)
|
||||
for entry in addr_info:
|
||||
ip = ipaddress.ip_address(entry[4][0])
|
||||
if ip.is_private or ip.is_loopback or ip.is_link_local or ip.is_reserved or ip.is_multicast:
|
||||
return False
|
||||
return True
|
||||
except Exception:
|
||||
return False # Block on resolution failure
|
||||
|
||||
_ALLOWED_TYPES = {
|
||||
"image/jpeg",
|
||||
"image/png",
|
||||
@@ -59,6 +82,10 @@ async def fetch_and_store_image(
|
||||
Returns None if the URL is unreachable, not a valid image type, or
|
||||
exceeds IMAGE_MAX_BYTES.
|
||||
"""
|
||||
if not _is_safe_image_url(url):
|
||||
logger.warning("Blocked image fetch of private/internal URL: %s", url[:80])
|
||||
return None
|
||||
|
||||
url_hash = _url_hash(url)
|
||||
|
||||
# Return existing record if already cached
|
||||
|
||||
@@ -454,6 +454,8 @@ async def build_context(
|
||||
workspace_project_id: int | None = None,
|
||||
user_timezone: str | None = None,
|
||||
conv_id: int | None = None,
|
||||
voice_mode: bool = False,
|
||||
voice_speech_style: str = "conversational",
|
||||
) -> tuple[list[dict], dict]:
|
||||
"""Build messages array for Ollama with system prompt and context.
|
||||
|
||||
@@ -516,6 +518,19 @@ async def build_context(
|
||||
f"{tool_guidance}"
|
||||
]
|
||||
|
||||
if voice_mode:
|
||||
_style_hints = {
|
||||
"conversational": "Be warm, natural, and conversational — like speaking to a friend.",
|
||||
"concise": "Be brief and to the point. One or two sentences maximum unless detail is essential.",
|
||||
"detailed": "Give thorough, informative responses as if narrating an explanation aloud.",
|
||||
}
|
||||
style_hint = _style_hints.get(voice_speech_style, _style_hints["conversational"])
|
||||
system_parts.insert(0,
|
||||
"VOICE MODE: Respond naturally as if speaking aloud. "
|
||||
"No markdown, bullet points, headers, or code blocks. Complete sentences only. "
|
||||
f"{style_hint}\n\n"
|
||||
)
|
||||
|
||||
context_meta: dict = {
|
||||
"context_note_id": None,
|
||||
"context_note_title": None,
|
||||
|
||||
@@ -17,6 +17,7 @@ async def create_milestone(
|
||||
title: str,
|
||||
description: str | None = None,
|
||||
order_index: int = 0,
|
||||
status: str = "active",
|
||||
) -> Milestone:
|
||||
async with async_session() as session:
|
||||
milestone = Milestone(
|
||||
@@ -24,7 +25,7 @@ async def create_milestone(
|
||||
project_id=project_id,
|
||||
title=title,
|
||||
description=description,
|
||||
status="active",
|
||||
status=status,
|
||||
order_index=order_index,
|
||||
)
|
||||
session.add(milestone)
|
||||
|
||||
@@ -17,6 +17,7 @@ async def create_project(
|
||||
description: str = "",
|
||||
goal: str = "",
|
||||
color: str | None = None,
|
||||
status: str = "active",
|
||||
) -> Project:
|
||||
async with async_session() as session:
|
||||
project = Project(
|
||||
@@ -25,7 +26,7 @@ async def create_project(
|
||||
description=description,
|
||||
goal=goal,
|
||||
color=color,
|
||||
status="active",
|
||||
status=status,
|
||||
)
|
||||
session.add(project)
|
||||
await session.commit()
|
||||
|
||||
@@ -55,6 +55,10 @@ async def fetch_and_cache_feed(feed_id: int, url: str) -> int:
|
||||
Fetch a feed URL, parse it, and upsert new items into rss_items.
|
||||
Returns the number of new items stored.
|
||||
"""
|
||||
scheme = url.split("://")[0].lower() if "://" in url else ""
|
||||
if scheme not in ("http", "https"):
|
||||
logger.warning("Blocked RSS fetch with non-http(s) scheme: %s", url[:80])
|
||||
return 0
|
||||
try:
|
||||
parsed = await _parse_feed(url)
|
||||
except Exception:
|
||||
|
||||
@@ -0,0 +1,82 @@
|
||||
"""Speech-to-text service using faster-whisper (in-process, CPU/GPU)."""
|
||||
import asyncio
|
||||
import logging
|
||||
import tempfile
|
||||
import time
|
||||
from typing import TYPE_CHECKING
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from faster_whisper import WhisperModel
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
_model: "WhisperModel | None" = None
|
||||
_model_lock = asyncio.Lock()
|
||||
_load_error: str | None = None
|
||||
|
||||
|
||||
async def load_stt_model() -> None:
|
||||
"""Load the Whisper model. Called once at startup when voice is enabled."""
|
||||
global _model, _load_error
|
||||
from fabledassistant.services.voice_config import get_stt_model, is_voice_enabled
|
||||
|
||||
if not await is_voice_enabled():
|
||||
return
|
||||
|
||||
async with _model_lock:
|
||||
if _model is not None:
|
||||
return
|
||||
try:
|
||||
from faster_whisper import WhisperModel
|
||||
|
||||
model_name = await get_stt_model()
|
||||
logger.info("Loading Whisper STT model '%s'...", model_name)
|
||||
loop = asyncio.get_running_loop()
|
||||
_model = await loop.run_in_executor(
|
||||
None,
|
||||
lambda: WhisperModel(model_name, device="cpu", compute_type="int8"),
|
||||
)
|
||||
logger.info("Whisper STT model '%s' loaded", model_name)
|
||||
except Exception:
|
||||
_load_error = "Failed to load Whisper STT model"
|
||||
logger.exception(_load_error)
|
||||
|
||||
|
||||
async def reload_stt_model() -> None:
|
||||
"""Unload the current model and reload with the current config. Safe to call at runtime."""
|
||||
global _model, _load_error
|
||||
async with _model_lock:
|
||||
_model = None
|
||||
_load_error = None
|
||||
await load_stt_model()
|
||||
|
||||
|
||||
def stt_available() -> bool:
|
||||
return _model is not None
|
||||
|
||||
|
||||
async def transcribe(audio_bytes: bytes, mime_type: str = "audio/webm") -> str:
|
||||
"""Transcribe audio bytes to text. Runs the model in a thread executor."""
|
||||
if _model is None:
|
||||
raise RuntimeError("STT model not loaded")
|
||||
|
||||
suffix = ".webm"
|
||||
if "ogg" in mime_type:
|
||||
suffix = ".ogg"
|
||||
elif "wav" in mime_type:
|
||||
suffix = ".wav"
|
||||
elif "mp4" in mime_type or "m4a" in mime_type:
|
||||
suffix = ".mp4"
|
||||
|
||||
def _run() -> str:
|
||||
with tempfile.NamedTemporaryFile(suffix=suffix, delete=True) as f:
|
||||
f.write(audio_bytes)
|
||||
f.flush()
|
||||
t0 = time.monotonic()
|
||||
segments, _ = _model.transcribe(f.name, beam_size=5) # type: ignore[union-attr]
|
||||
text = " ".join(seg.text.strip() for seg in segments).strip()
|
||||
logger.debug("STT transcription took %.2fs", time.monotonic() - t0)
|
||||
return text
|
||||
|
||||
loop = asyncio.get_running_loop()
|
||||
return await loop.run_in_executor(None, _run)
|
||||
@@ -0,0 +1,103 @@
|
||||
"""Text-to-speech service using Kokoro TTS (in-process)."""
|
||||
import asyncio
|
||||
import io
|
||||
import logging
|
||||
import time
|
||||
from typing import TYPE_CHECKING
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from kokoro import KPipeline
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
_pipeline: "KPipeline | None" = None
|
||||
_pipeline_lock = asyncio.Lock()
|
||||
_load_error: str | None = None
|
||||
|
||||
# Static list of supported Kokoro voice IDs and display labels
|
||||
_VOICES: list[dict] = [
|
||||
{"id": "af_heart", "label": "Heart (American Female, warm)"},
|
||||
{"id": "af_bella", "label": "Bella (American Female, expressive)"},
|
||||
{"id": "af_nicole", "label": "Nicole (American Female, intimate)"},
|
||||
{"id": "af_sarah", "label": "Sarah (American Female, clear)"},
|
||||
{"id": "af_sky", "label": "Sky (American Female, bright)"},
|
||||
{"id": "am_adam", "label": "Adam (American Male, neutral)"},
|
||||
{"id": "am_michael", "label": "Michael (American Male, deep)"},
|
||||
{"id": "bf_emma", "label": "Emma (British Female)"},
|
||||
{"id": "bf_isabella", "label": "Isabella (British Female, formal)"},
|
||||
{"id": "bm_george", "label": "George (British Male)"},
|
||||
{"id": "bm_lewis", "label": "Lewis (British Male, casual)"},
|
||||
]
|
||||
|
||||
|
||||
async def load_tts_model() -> None:
|
||||
"""Load the Kokoro pipeline. Called once at startup when voice is enabled."""
|
||||
global _pipeline, _load_error
|
||||
from fabledassistant.services.voice_config import is_voice_enabled
|
||||
|
||||
if not await is_voice_enabled():
|
||||
return
|
||||
|
||||
async with _pipeline_lock:
|
||||
if _pipeline is not None:
|
||||
return
|
||||
try:
|
||||
from kokoro import KPipeline
|
||||
|
||||
logger.info("Loading Kokoro TTS pipeline...")
|
||||
loop = asyncio.get_running_loop()
|
||||
_pipeline = await loop.run_in_executor(
|
||||
None,
|
||||
lambda: KPipeline(lang_code="a"), # "a" = American English
|
||||
)
|
||||
logger.info("Kokoro TTS pipeline loaded")
|
||||
except Exception:
|
||||
_load_error = "Failed to load Kokoro TTS pipeline"
|
||||
logger.exception(_load_error)
|
||||
|
||||
|
||||
async def reload_tts_model() -> None:
|
||||
"""Unload and reload the TTS pipeline. Safe to call at runtime."""
|
||||
global _pipeline, _load_error
|
||||
async with _pipeline_lock:
|
||||
_pipeline = None
|
||||
_load_error = None
|
||||
await load_tts_model()
|
||||
|
||||
|
||||
def tts_available() -> bool:
|
||||
return _pipeline is not None
|
||||
|
||||
|
||||
def list_voices() -> list[dict]:
|
||||
return _VOICES
|
||||
|
||||
|
||||
async def synthesise(text: str, voice: str = "af_heart", speed: float = 1.0) -> bytes:
|
||||
"""Synthesise text to WAV bytes (24kHz, 16-bit mono). Runs in executor."""
|
||||
if _pipeline is None:
|
||||
raise RuntimeError("TTS pipeline not loaded")
|
||||
|
||||
speed = max(0.7, min(1.3, speed))
|
||||
|
||||
def _run() -> bytes:
|
||||
import numpy as np
|
||||
import soundfile as sf
|
||||
|
||||
t0 = time.monotonic()
|
||||
audio_chunks: list = []
|
||||
for _, _, audio in _pipeline(text, voice=voice, speed=speed): # type: ignore[misc]
|
||||
if audio is not None:
|
||||
audio_chunks.append(audio)
|
||||
|
||||
if not audio_chunks:
|
||||
return b""
|
||||
|
||||
combined = np.concatenate(audio_chunks)
|
||||
buf = io.BytesIO()
|
||||
sf.write(buf, combined, samplerate=24000, format="WAV", subtype="PCM_16")
|
||||
logger.debug("TTS synthesis took %.2fs for %d chars", time.monotonic() - t0, len(text))
|
||||
return buf.getvalue()
|
||||
|
||||
loop = asyncio.get_running_loop()
|
||||
return await loop.run_in_executor(None, _run)
|
||||
@@ -0,0 +1,40 @@
|
||||
"""System-wide voice configuration stored in the admin user's settings row.
|
||||
|
||||
Falls back to Config env vars so operators can still seed defaults via the
|
||||
environment without touching the UI.
|
||||
"""
|
||||
from sqlalchemy import select
|
||||
|
||||
from fabledassistant.config import Config
|
||||
from fabledassistant.models import async_session
|
||||
from fabledassistant.models.setting import Setting
|
||||
from fabledassistant.models.user import User
|
||||
|
||||
VOICE_SETTING_KEYS = ("voice_enabled", "voice_stt_model")
|
||||
|
||||
|
||||
async def get_voice_config() -> dict[str, str]:
|
||||
"""Return voice config, preferring DB values over env-var defaults."""
|
||||
config: dict[str, str] = {
|
||||
"voice_enabled": "true" if Config.VOICE_ENABLED else "false",
|
||||
"voice_stt_model": Config.STT_MODEL,
|
||||
}
|
||||
async with async_session() as session:
|
||||
result = await session.execute(
|
||||
select(Setting)
|
||||
.join(User, Setting.user_id == User.id)
|
||||
.where(User.role == "admin", Setting.key.in_(VOICE_SETTING_KEYS))
|
||||
)
|
||||
for setting in result.scalars().all():
|
||||
config[setting.key] = setting.value
|
||||
return config
|
||||
|
||||
|
||||
async def is_voice_enabled() -> bool:
|
||||
config = await get_voice_config()
|
||||
return config.get("voice_enabled", "false").lower() in ("1", "true", "yes")
|
||||
|
||||
|
||||
async def get_stt_model() -> str:
|
||||
config = await get_voice_config()
|
||||
return config.get("voice_stt_model", Config.STT_MODEL)
|
||||
Reference in New Issue
Block a user