docs: add web voice overlay polish spec

2026-04-06 19:39:27 -04:00
parent edf0a9063e
commit 1b68559bfe
1 changed files with 278 additions and 0 deletions
@@ -0,0 +1,278 @@
+# Web Voice Overlay Polish — Implementation Spec
+
+> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
+
+**Goal:** Ship the dormant `VoiceOverlay` component by mounting it, wiring the Space bar shortcut, and replacing push-to-talk with click-to-toggle silence detection.
+
+**Architecture:** A new `useSilenceDetector` composable wraps the Web Audio API `AnalyserNode` and fires a callback when sustained silence is detected. `VoiceOverlay` coordinates `useVoiceRecorder` and `useSilenceDetector`, switching from hold-to-record to click-to-toggle. `App.vue` mounts the overlay and adds the Space bar handler.
+
+**Tech Stack:** Vue 3 Composition API, Web Audio API (`AnalyserNode`), existing `useVoiceRecorder` / `useVoiceAudio` composables, TypeScript.
+
+---
+
+## File Map
+
+| Action | Path |
+|--------|------|
+| Create | `frontend/src/composables/useSilenceDetector.ts` |
+| Modify | `frontend/src/composables/useVoiceRecorder.ts` |
+| Modify | `frontend/src/components/VoiceOverlay.vue` |
+| Modify | `frontend/src/App.vue` |
+
+---
+
+## Task 1: `useSilenceDetector` composable
+
+**Files:**
+- Create: `frontend/src/composables/useSilenceDetector.ts`
+
+### Interface
+
+```ts
+export interface SilenceDetectorOptions {
+  thresholdDb?: number       // default -40
+  silenceDurationMs?: number // default 1500
+  minRecordingMs?: number    // default 500
+}
+
+export function useSilenceDetector(options?: SilenceDetectorOptions): {
+  amplitude: Readonly<Ref<number>>   // 0–1, for visualization
+  start(stream: MediaStream, onSilence: () => void): void
+  stop(): void
+}
+```
+
+### Behaviour
+
+- `start(stream, onSilence)`:
+  1. Creates `AudioContext`
+  2. `createMediaStreamSource(stream)` → connects to `AnalyserNode` (fftSize 256)
+  3. Records `startedAt = Date.now()`
+  4. Starts a `setInterval` at 100ms that:
+     - Calls `analyser.getByteFrequencyData(dataArray)`
+     - Computes RMS amplitude → maps to 0–1 range for `amplitude.value`
+     - Converts to approximate dB: `db = 20 * log10(rms)` (clamp to -100 when rms === 0)
+     - If `db < thresholdDb`: increments `silenceMs += 100`; else resets `silenceMs = 0`
+     - If `silenceMs >= silenceDurationMs` AND `Date.now() - startedAt >= minRecordingMs`: clears interval, fires `onSilence()`
+- `stop()`: clears interval, closes `AudioContext`, resets `amplitude.value = 0`
+- Safe to call `stop()` multiple times (guard with null check)
+- `amplitude` resets to 0 after `stop()`
+
+### Full implementation
+
+```ts
+import { ref, readonly } from 'vue'
+
+export interface SilenceDetectorOptions {
+  thresholdDb?: number
+  silenceDurationMs?: number
+  minRecordingMs?: number
+}
+
+export function useSilenceDetector(options: SilenceDetectorOptions = {}) {
+  const {
+    thresholdDb = -40,
+    silenceDurationMs = 1500,
+    minRecordingMs = 500,
+  } = options
+
+  const amplitude = ref(0)
+  let audioCtx: AudioContext | null = null
+  let intervalId: ReturnType<typeof setInterval> | null = null
+  let silenceMs = 0
+  let startedAt = 0
+
+  function start(stream: MediaStream, onSilence: () => void) {
+    stop()
+    audioCtx = new AudioContext()
+    const source = audioCtx.createMediaStreamSource(stream)
+    const analyser = audioCtx.createAnalyser()
+    analyser.fftSize = 256
+    source.connect(analyser)
+
+    const data = new Uint8Array(analyser.frequencyBinCount)
+    silenceMs = 0
+    startedAt = Date.now()
+
+    intervalId = setInterval(() => {
+      analyser.getByteFrequencyData(data)
+      const rms = Math.sqrt(data.reduce((s, v) => s + v * v, 0) / data.length) / 255
+      amplitude.value = rms
+
+      const db = rms > 0 ? 20 * Math.log10(rms) : -100
+      if (db < thresholdDb) {
+        silenceMs += 100
+        if (silenceMs >= silenceDurationMs && Date.now() - startedAt >= minRecordingMs) {
+          stop()
+          onSilence()
+        }
+      } else {
+        silenceMs = 0
+      }
+    }, 100)
+  }
+
+  function stop() {
+    if (intervalId !== null) {
+      clearInterval(intervalId)
+      intervalId = null
+    }
+    if (audioCtx) {
+      audioCtx.close().catch(() => {})
+      audioCtx = null
+    }
+    amplitude.value = 0
+    silenceMs = 0
+  }
+
+  return { amplitude: readonly(amplitude), start, stop }
+}
+```
+
+- [ ] Write the file exactly as above
+- [ ] Verify TypeScript compiles: `cd frontend && npx tsc --noEmit`
+- [ ] Commit: `git add frontend/src/composables/useSilenceDetector.ts && git commit -m "feat: add useSilenceDetector composable"`
+
+---
+
+## Task 2: Expose `stream` from `useVoiceRecorder`
+
+**Files:**
+- Modify: `frontend/src/composables/useVoiceRecorder.ts`
+
+Change the `stream` local variable to a `Ref<MediaStream | null>` and export it as readonly.
+
+- [ ] Change `let stream: MediaStream | null = null` to `const streamRef = ref<MediaStream | null>(null)`
+- [ ] Replace all `stream` assignments with `streamRef.value`:
+  - `stream = await navigator.mediaDevices.getUserMedia(...)` → `streamRef.value = await ...`
+  - `stream?.getTracks().forEach(...)` → `streamRef.value?.getTracks().forEach(...)`
+  - `stream = null` → `streamRef.value = null`
+- [ ] Add `stream: readonly(streamRef)` to the return object
+- [ ] Verify TypeScript: `npx tsc --noEmit`
+- [ ] Commit: `git add frontend/src/composables/useVoiceRecorder.ts && git commit -m "feat: expose stream ref from useVoiceRecorder"`
+
+---
+
+## Task 3: Wire `VoiceOverlay` — silence detection + click-to-toggle
+
+**Files:**
+- Modify: `frontend/src/components/VoiceOverlay.vue`
+
+### Script changes
+
+- [ ] Import `useSilenceDetector` at the top of `<script setup>`
+- [ ] Add `const silenceDetector = useSilenceDetector()` after the existing composable instantiations
+- [ ] In `startPtt()`: after `phase.value = 'recording'`, add:
+  ```ts
+  if (recorder.stream.value) {
+    silenceDetector.start(recorder.stream.value, stopPtt)
+  }
+  ```
+- [ ] In `stopPtt()`: add `silenceDetector.stop()` as the first line (before the guard check)
+- [ ] In `cancelAll()`: add `silenceDetector.stop()` after `recorder.stopRecording().catch(() => {})`
+
+### Button: click-to-toggle
+
+Replace the PTT mouse/touch handlers on `.voice-ptt-btn` with click-to-toggle logic:
+
+- [ ] Remove `@mousedown.prevent="startPtt"` and `@mouseup.prevent="stopPtt"`
+- [ ] Remove `@touchstart.prevent="startPtt"` and `@touchend.prevent="stopPtt"`
+- [ ] Replace `@click.prevent="phase === 'error' ? (phase = 'idle') : undefined"` with:
+  ```html
+  @click.prevent="onBtnClick"
+  ```
+- [ ] Add `onBtnClick` function in script:
+  ```ts
+  function onBtnClick() {
+    if (phase.value === 'error') { phase.value = 'idle'; return }
+    if (phase.value === 'recording') { stopPtt(); return }
+    if (phase.value === 'idle') { startPtt() }
+  }
+  ```
+- [ ] Update `aria-label` and `title` on the button:
+  - `aria-label`: `phase === 'recording' ? 'Click to stop' : 'Click to speak'`
+  - `title`: `phase === 'recording' ? 'Click to stop or wait for silence' : 'Click or press Space to speak'`
+
+### Amplitude visualization during recording
+
+Inside the button, when `phase === 'recording'`, replace the static stop icon with animated amplitude bars:
+
+- [ ] Replace the recording SVG block:
+  ```html
+  <svg v-else-if="phase === 'recording'" ...>...</svg>
+  ```
+  with:
+  ```html
+  <span v-else-if="phase === 'recording'" class="voice-amp-bars">
+    <span
+      v-for="n in 3"
+      :key="n"
+      class="voice-amp-bar"
+      :style="{ transform: `scaleY(${0.3 + silenceDetector.amplitude.value * (0.4 + n * 0.15)})` }"
+    ></span>
+  </span>
+  ```
+
+### Hint label
+
+- [ ] Change the idle hint from `Hold <kbd>Space</kbd> or tap` to `Tap or press <kbd>Space</kbd>`
+
+### CSS for amplitude bars
+
+- [ ] Add to `<style scoped>`:
+  ```css
+  .voice-amp-bars {
+    display: flex;
+    gap: 3px;
+    align-items: center;
+    height: 22px;
+  }
+  .voice-amp-bar {
+    width: 4px;
+    height: 18px;
+    background: #fff;
+    border-radius: 2px;
+    transform-origin: center;
+    transition: transform 0.08s ease;
+  }
+  ```
+
+- [ ] Verify TypeScript: `npx tsc --noEmit`
+- [ ] Commit: `git add frontend/src/components/VoiceOverlay.vue && git commit -m "feat: click-to-toggle silence detection in VoiceOverlay"`
+
+---
+
+## Task 4: Mount overlay and wire Space bar in `App.vue`
+
+**Files:**
+- Modify: `frontend/src/App.vue`
+
+### Mount VoiceOverlay
+
+- [ ] Add import at top of `<script setup>`:
+  ```ts
+  import VoiceOverlay from '@/components/VoiceOverlay.vue'
+  ```
+- [ ] Add `<VoiceOverlay />` inside the `<template v-if="authStore.isAuthenticated">` block, just before `<ToastNotification />`:
+  ```html
+  <VoiceOverlay />
+  <ToastNotification />
+  ```
+
+### Space bar handler
+
+- [ ] In `onGlobalKeydown`, add a `Space` case inside the `switch (e.key)` block (after the existing cases), only fires when `!isInputActive()`:
+  ```ts
+  case ' ':
+    e.preventDefault()
+    document.dispatchEvent(new CustomEvent('voice:ptt-toggle'))
+    break
+  ```
+
+### Shortcuts panel label
+
+- [ ] Update the Space shortcut description from `Hold to speak (voice, when enabled)` to `Tap to speak (voice, when enabled)`
+
+- [ ] Verify TypeScript: `npx tsc --noEmit`
+- [ ] Verify full build: `npm run build`
+- [ ] Commit: `git add frontend/src/App.vue && git commit -m "feat: mount VoiceOverlay and wire Space bar shortcut"`