FabledScribe/docs/superpowers/specs/2026-04-06-web-voice-overlay-polish.md

# Web Voice Overlay Polish — Implementation Spec

> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.

**Goal:** Ship the dormant `VoiceOverlay` component by mounting it, wiring the Space bar shortcut, and replacing push-to-talk with click-to-toggle silence detection.

**Architecture:** A new `useSilenceDetector` composable wraps the Web Audio API `AnalyserNode` and fires a callback when sustained silence is detected. `VoiceOverlay` coordinates `useVoiceRecorder` and `useSilenceDetector`, switching from hold-to-record to click-to-toggle. `App.vue` mounts the overlay and adds the Space bar handler.

**Tech Stack:** Vue 3 Composition API, Web Audio API (`AnalyserNode`), existing `useVoiceRecorder` / `useVoiceAudio` composables, TypeScript.

---

## File Map

| Action | Path |
|--------|------|
| Create | `frontend/src/composables/useSilenceDetector.ts` |
| Modify | `frontend/src/composables/useVoiceRecorder.ts` |
| Modify | `frontend/src/components/VoiceOverlay.vue` |
| Modify | `frontend/src/App.vue` |

---

## Task 1: `useSilenceDetector` composable

**Files:**
- Create: `frontend/src/composables/useSilenceDetector.ts`

### Interface

```ts
export interface SilenceDetectorOptions {
  thresholdDb?: number       // default -40
  silenceDurationMs?: number // default 1500
  minRecordingMs?: number    // default 500
}

export function useSilenceDetector(options?: SilenceDetectorOptions): {
  amplitude: Readonly<Ref<number>>   // 0–1, for visualization
  start(stream: MediaStream, onSilence: () => void): void
  stop(): void
}
```

### Behaviour

- `start(stream, onSilence)`:
  1. Creates `AudioContext`
  2. `createMediaStreamSource(stream)` → connects to `AnalyserNode` (fftSize 256)
  3. Records `startedAt = Date.now()`
  4. Starts a `setInterval` at 100ms that:
     - Calls `analyser.getByteFrequencyData(dataArray)`
     - Computes RMS amplitude → maps to 0–1 range for `amplitude.value`
     - Converts to approximate dB: `db = 20 * log10(rms)` (clamp to -100 when rms === 0)
     - If `db < thresholdDb`: increments `silenceMs += 100`; else resets `silenceMs = 0`
     - If `silenceMs >= silenceDurationMs` AND `Date.now() - startedAt >= minRecordingMs`: clears interval, fires `onSilence()`
- `stop()`: clears interval, closes `AudioContext`, resets `amplitude.value = 0`
- Safe to call `stop()` multiple times (guard with null check)
- `amplitude` resets to 0 after `stop()`

### Full implementation

```ts
import { ref, readonly } from 'vue'

export interface SilenceDetectorOptions {
  thresholdDb?: number
  silenceDurationMs?: number
  minRecordingMs?: number
}

export function useSilenceDetector(options: SilenceDetectorOptions = {}) {
  const {
    thresholdDb = -40,
    silenceDurationMs = 1500,
    minRecordingMs = 500,
  } = options

  const amplitude = ref(0)
  let audioCtx: AudioContext | null = null
  let intervalId: ReturnType<typeof setInterval> | null = null
  let silenceMs = 0
  let startedAt = 0

  function start(stream: MediaStream, onSilence: () => void) {
    stop()
    audioCtx = new AudioContext()
    const source = audioCtx.createMediaStreamSource(stream)
    const analyser = audioCtx.createAnalyser()
    analyser.fftSize = 256
    source.connect(analyser)

    const data = new Uint8Array(analyser.frequencyBinCount)
    silenceMs = 0
    startedAt = Date.now()

    intervalId = setInterval(() => {
      analyser.getByteFrequencyData(data)
      const rms = Math.sqrt(data.reduce((s, v) => s + v * v, 0) / data.length) / 255
      amplitude.value = rms

      const db = rms > 0 ? 20 * Math.log10(rms) : -100
      if (db < thresholdDb) {
        silenceMs += 100
        if (silenceMs >= silenceDurationMs && Date.now() - startedAt >= minRecordingMs) {
          stop()
          onSilence()
        }
      } else {
        silenceMs = 0
      }
    }, 100)
  }

  function stop() {
    if (intervalId !== null) {
      clearInterval(intervalId)
      intervalId = null
    }
    if (audioCtx) {
      audioCtx.close().catch(() => {})
      audioCtx = null
    }
    amplitude.value = 0
    silenceMs = 0
  }

  return { amplitude: readonly(amplitude), start, stop }
}
```

- [ ] Write the file exactly as above
- [ ] Verify TypeScript compiles: `cd frontend && npx tsc --noEmit`
- [ ] Commit: `git add frontend/src/composables/useSilenceDetector.ts && git commit -m "feat: add useSilenceDetector composable"`

---

## Task 2: Expose `stream` from `useVoiceRecorder`

**Files:**
- Modify: `frontend/src/composables/useVoiceRecorder.ts`

Change the `stream` local variable to a `Ref<MediaStream | null>` and export it as readonly.

- [ ] Change `let stream: MediaStream | null = null` to `const streamRef = ref<MediaStream | null>(null)`
- [ ] Replace all `stream` assignments with `streamRef.value`:
  - `stream = await navigator.mediaDevices.getUserMedia(...)` → `streamRef.value = await ...`
  - `stream?.getTracks().forEach(...)` → `streamRef.value?.getTracks().forEach(...)`
  - `stream = null` → `streamRef.value = null`
- [ ] Add `stream: readonly(streamRef)` to the return object
- [ ] Verify TypeScript: `npx tsc --noEmit`
- [ ] Commit: `git add frontend/src/composables/useVoiceRecorder.ts && git commit -m "feat: expose stream ref from useVoiceRecorder"`

---

## Task 3: Wire `VoiceOverlay` — silence detection + click-to-toggle

**Files:**
- Modify: `frontend/src/components/VoiceOverlay.vue`

### Script changes

- [ ] Import `useSilenceDetector` at the top of `<script setup>`
- [ ] Add `const silenceDetector = useSilenceDetector()` after the existing composable instantiations
- [ ] In `startPtt()`: after `phase.value = 'recording'`, add:
  ```ts
  if (recorder.stream.value) {
    silenceDetector.start(recorder.stream.value, stopPtt)
  }
  ```
- [ ] In `stopPtt()`: add `silenceDetector.stop()` as the first line (before the guard check)
- [ ] In `cancelAll()`: add `silenceDetector.stop()` after `recorder.stopRecording().catch(() => {})`

### Button: click-to-toggle

Replace the PTT mouse/touch handlers on `.voice-ptt-btn` with click-to-toggle logic:

- [ ] Remove `@mousedown.prevent="startPtt"` and `@mouseup.prevent="stopPtt"`
- [ ] Remove `@touchstart.prevent="startPtt"` and `@touchend.prevent="stopPtt"`
- [ ] Replace `@click.prevent="phase === 'error' ? (phase = 'idle') : undefined"` with:
  ```html
  @click.prevent="onBtnClick"
  ```
- [ ] Add `onBtnClick` function in script:
  ```ts
  function onBtnClick() {
    if (phase.value === 'error') { phase.value = 'idle'; return }
    if (phase.value === 'recording') { stopPtt(); return }
    if (phase.value === 'idle') { startPtt() }
  }
  ```
- [ ] Update `aria-label` and `title` on the button:
  - `aria-label`: `phase === 'recording' ? 'Click to stop' : 'Click to speak'`
  - `title`: `phase === 'recording' ? 'Click to stop or wait for silence' : 'Click or press Space to speak'`

### Amplitude visualization during recording

Inside the button, when `phase === 'recording'`, replace the static stop icon with animated amplitude bars:

- [ ] Replace the recording SVG block:
  ```html
  <svg v-else-if="phase === 'recording'" ...>...</svg>
  ```
  with:
  ```html
  <span v-else-if="phase === 'recording'" class="voice-amp-bars">
    <span
      v-for="n in 3"
      :key="n"
      class="voice-amp-bar"
      :style="{ transform: `scaleY(${0.3 + silenceDetector.amplitude.value * (0.4 + n * 0.15)})` }"
    ></span>
  </span>
  ```

### Hint label

- [ ] Change the idle hint from `Hold <kbd>Space</kbd> or tap` to `Tap or press <kbd>Space</kbd>`

### CSS for amplitude bars

- [ ] Add to `<style scoped>`:
  ```css
  .voice-amp-bars {
    display: flex;
    gap: 3px;
    align-items: center;
    height: 22px;
  }
  .voice-amp-bar {
    width: 4px;
    height: 18px;
    background: #fff;
    border-radius: 2px;
    transform-origin: center;
    transition: transform 0.08s ease;
  }
  ```

- [ ] Verify TypeScript: `npx tsc --noEmit`
- [ ] Commit: `git add frontend/src/components/VoiceOverlay.vue && git commit -m "feat: click-to-toggle silence detection in VoiceOverlay"`

---

## Task 4: Mount overlay and wire Space bar in `App.vue`

**Files:**
- Modify: `frontend/src/App.vue`

### Mount VoiceOverlay

- [ ] Add import at top of `<script setup>`:
  ```ts
  import VoiceOverlay from '@/components/VoiceOverlay.vue'
  ```
- [ ] Add `<VoiceOverlay />` inside the `<template v-if="authStore.isAuthenticated">` block, just before `<ToastNotification />`:
  ```html
  <VoiceOverlay />
  <ToastNotification />
  ```

### Space bar handler

- [ ] In `onGlobalKeydown`, add a `Space` case inside the `switch (e.key)` block (after the existing cases), only fires when `!isInputActive()`:
  ```ts
  case ' ':
    e.preventDefault()
    document.dispatchEvent(new CustomEvent('voice:ptt-toggle'))
    break
  ```

### Shortcuts panel label

- [ ] Update the Space shortcut description from `Hold to speak (voice, when enabled)` to `Tap to speak (voice, when enabled)`

- [ ] Verify TypeScript: `npx tsc --noEmit`
- [ ] Verify full build: `npm run build`
- [ ] Commit: `git add frontend/src/App.vue && git commit -m "feat: mount VoiceOverlay and wire Space bar shortcut"`