FabledApp/docs/superpowers/plans/2026-04-05-android-voice-io.md

Android Voice I/O Implementation Plan

For agentic workers: REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (- [ ]) syntax for tracking.

Goal: Add server-side STT and streaming TTS to the Android app — mic button in Chat, Quick Capture bar, and Briefing.

Architecture: A shared VoiceNotifier (Riverpod Notifier) owns the recording lifecycle, silence detection, and TTS sentence queue. Screens pass an onTranscript callback when entering voice mode; TTS is fed back sentence-by-sentence via feedContent() as the SSE response streams in. All audio I/O goes through the existing /api/voice/* endpoints — no backend changes needed.

Tech Stack: Flutter/Dart, record ^5.0.0 (WebM/Opus recording), just_audio ^0.9.x (WAV playback), permission_handler ^11.x (already in pubspec), path_provider ^2.x (already in pubspec), Riverpod 3, Dio.

---

File map

Action	Path	Responsibility
Create	lib/data/api/voice_api.dart	Dio calls: checkStatus, transcribe, synthesise
Create	lib/data/repositories/voice_repository.dart	Thin wrapper around VoiceApi
Modify	lib/providers/api_client_provider.dart	Add voiceApiProvider, voiceRepositoryProvider
Create	lib/providers/voice_provider.dart	VoiceState + VoiceNotifier (recording, silence, TTS)
Create	lib/widgets/voice_mic_button.dart	Animated mic button widget
Modify	lib/screens/chat/chat_screen.dart	Add mic button + voice banner + TTS feed
Modify	lib/app.dart	Add mic button to _QuickCaptureBar
Modify	lib/screens/briefing/briefing_screen.dart	Add mic button + voice banner + TTS feed
Modify	pubspec.yaml	Add record, just_audio
Modify	android/app/src/main/AndroidManifest.xml	RECORD_AUDIO permission
Modify	test/widget_test.dart	Unit tests for pure voice logic

---

Task 1: Dependencies and permissions

Files:

• Modify: pubspec.yaml

• Modify: android/app/src/main/AndroidManifest.xml

• Step 1: Add packages to pubspec.yaml

In pubspec.yaml, add under dependencies: (after url_launcher):

  record: ^5.0.0
  just_audio: ^0.9.39

• Step 2: Add RECORD_AUDIO permission to AndroidManifest.xml

In android/app/src/main/AndroidManifest.xml, add after the existing <uses-permission android:name="android.permission.ACCESS_NETWORK_STATE"/> line:

    <uses-permission android:name="android.permission.RECORD_AUDIO"/>

• Step 3: Install packages

cd /home/bvandeusen/Nextcloud/Projects/fabled_app
flutter pub get

Expected: resolves record 5.x and just_audio 0.9.x without conflicts.

• Step 4: Verify no compile errors

flutter analyze

Expected: no new errors.

• Step 5: Commit

git add pubspec.yaml pubspec.lock android/app/src/main/AndroidManifest.xml
git commit -m "feat(voice): add record + just_audio dependencies, RECORD_AUDIO permission"

---

Task 2: VoiceApi and VoiceStatus model

Files:

• Create: lib/data/api/voice_api.dart

• Modify: test/widget_test.dart

• Step 1: Write failing tests for VoiceStatus.fromJson

Add to test/widget_test.dart:

import 'package:fabled_app/data/api/voice_api.dart';

Add a new group at the bottom of main():

  group('VoiceStatus.fromJson', () {
    test('parses enabled with stt and tts available', () {
      final status = VoiceStatus.fromJson({
        'enabled': true,
        'stt': true,
        'tts': true,
      });
      expect(status.enabled, isTrue);
      expect(status.stt, isTrue);
      expect(status.tts, isTrue);
    });

    test('parses disabled state', () {
      final status = VoiceStatus.fromJson({
        'enabled': false,
        'stt': false,
        'tts': false,
      });
      expect(status.enabled, isFalse);
    });

    test('fully unavailable when enabled but stt or tts false', () {
      final status = VoiceStatus.fromJson({
        'enabled': true,
        'stt': false,
        'tts': true,
      });
      expect(status.fullyAvailable, isFalse);
    });
  });

• Step 2: Run test to confirm it fails

flutter test test/widget_test.dart

Expected: FAIL — voice_api.dart not found.

• Step 3: Create lib/data/api/voice_api.dart

import 'dart:typed_data';

import 'package:dio/dio.dart';

import 'api_client.dart';

class VoiceStatus {
  final bool enabled;
  final bool stt;
  final bool tts;

  const VoiceStatus({
    required this.enabled,
    required this.stt,
    required this.tts,
  });

  /// True only when voice is enabled AND both STT and TTS are ready.
  bool get fullyAvailable => enabled && stt && tts;

  factory VoiceStatus.fromJson(Map<String, dynamic> json) => VoiceStatus(
        enabled: json['enabled'] as bool? ?? false,
        stt: json['stt'] as bool? ?? false,
        tts: json['tts'] as bool? ?? false,
      );
}

class VoiceApi {
  final Dio _dio;
  const VoiceApi(this._dio);

  /// Check whether voice features are available on this server.
  Future<VoiceStatus> checkStatus() async {
    try {
      final response = await _dio.get('/api/voice/status');
      return VoiceStatus.fromJson(response.data as Map<String, dynamic>);
    } on DioException catch (e) {
      throw dioToApp(e);
    }
  }

  /// POST WebM/Opus audio bytes and return the transcript string.
  /// Returns empty string on empty or error response.
  Future<String> transcribe(Uint8List audioBytes) async {
    try {
      final formData = FormData.fromMap({
        'audio': MultipartFile.fromBytes(
          audioBytes,
          filename: 'audio.webm',
          contentType: DioMediaType('audio', 'webm'),
        ),
      });
      final response = await _dio.post(
        '/api/voice/transcribe',
        data: formData,
        options: Options(
          receiveTimeout: const Duration(seconds: 60),
          contentType: 'multipart/form-data',
        ),
      );
      final data = response.data as Map<String, dynamic>;
      return (data['transcript'] as String? ?? '').trim();
    } on DioException catch (e) {
      throw dioToApp(e);
    }
  }

  /// POST text and return raw WAV bytes.
  Future<Uint8List> synthesise(String text) async {
    try {
      final response = await _dio.post(
        '/api/voice/synthesise',
        data: {'text': text},
        options: Options(responseType: ResponseType.bytes),
      );
      return Uint8List.fromList(response.data as List<int>);
    } on DioException catch (e) {
      throw dioToApp(e);
    }
  }
}

• Step 4: Run tests to confirm they pass

flutter test test/widget_test.dart

Expected: all tests pass including the new VoiceStatus group.

• Step 5: Commit

git add lib/data/api/voice_api.dart test/widget_test.dart
git commit -m "feat(voice): add VoiceApi and VoiceStatus model"

---

Task 3: VoiceRepository and provider registration

Files:

• Create: lib/data/repositories/voice_repository.dart

• Modify: lib/providers/api_client_provider.dart

• Step 1: Create lib/data/repositories/voice_repository.dart

import 'dart:typed_data';

import '../api/voice_api.dart';

class VoiceRepository {
  final VoiceApi _api;
  const VoiceRepository(this._api);

  Future<VoiceStatus> checkStatus() => _api.checkStatus();
  Future<String> transcribe(Uint8List audioBytes) => _api.transcribe(audioBytes);
  Future<Uint8List> synthesise(String text) => _api.synthesise(text);
}

• Step 2: Register providers in lib/providers/api_client_provider.dart

Add the following imports at the top of the file (after the existing imports):

import '../data/api/voice_api.dart';
import '../data/repositories/voice_repository.dart';

Add at the bottom of the file (after settingsApiProvider):

final voiceApiProvider = Provider<VoiceApi>((ref) {
  return VoiceApi(ref.watch(dioProvider));
});

final voiceRepositoryProvider = Provider<VoiceRepository>((ref) {
  return VoiceRepository(ref.watch(voiceApiProvider));
});

• Step 3: Verify no errors

flutter analyze

Expected: no errors.

• Step 4: Commit

git add lib/data/repositories/voice_repository.dart lib/providers/api_client_provider.dart
git commit -m "feat(voice): add VoiceRepository and provider registration"

---

Task 4: VoiceNotifier — state, recording, silence detection, transcription

Files:

• Create: lib/providers/voice_provider.dart

• Modify: test/widget_test.dart

• Step 1: Write failing tests for sentence extraction and markdown stripping

Add the following import to test/widget_test.dart:

import 'package:fabled_app/providers/voice_provider.dart';

Add a new group at the bottom of main():

  group('VoiceNotifier sentence extraction', () {
    test('extracts complete sentences at full stops', () {
      final result = extractSentences('Hello world. How are you? I am fine!');
      expect(result.sentences, equals(['Hello world.', 'How are you?', 'I am fine!']));
      expect(result.remainder, equals(''));
    });

    test('leaves incomplete fragment in remainder', () {
      final result = extractSentences('Hello world. Incomplete');
      expect(result.sentences, equals(['Hello world.']));
      expect(result.remainder, equals('Incomplete'));
    });

    test('returns empty sentences and full text when no boundary', () {
      final result = extractSentences('No boundary here');
      expect(result.sentences, isEmpty);
      expect(result.remainder, equals('No boundary here'));
    });
  });

  group('VoiceNotifier markdown stripping', () {
    test('strips code fences', () {
      expect(stripMarkdownForTts('Before\n```dart\ncode\n```\nAfter'), equals('Before After'));
    });

    test('strips bold and italic markers', () {
      expect(stripMarkdownForTts('**bold** and *italic*'), equals('bold and italic'));
    });

    test('strips headers', () {
      expect(stripMarkdownForTts('## Section title'), equals('Section title'));
    });

    test('keeps link text, removes URL', () {
      expect(stripMarkdownForTts('[click here](https://example.com)'), equals('click here'));
    });

    test('strips list markers', () {
      expect(stripMarkdownForTts('- item one\n- item two'), equals('item one item two'));
    });
  });

• Step 2: Run test to confirm failures

flutter test test/widget_test.dart

Expected: FAIL — voice_provider.dart not found.

• Step 3: Create lib/providers/voice_provider.dart

import 'dart:async';
import 'dart:collection';
import 'dart:io';
import 'dart:typed_data';

import 'package:flutter_riverpod/flutter_riverpod.dart';
import 'package:just_audio/just_audio.dart';
import 'package:path_provider/path_provider.dart';
import 'package:permission_handler/permission_handler.dart';
import 'package:record/record.dart';

import 'api_client_provider.dart';

// ── Public helpers (also used by tests) ──────────────────────────────────────

class SentenceResult {
  final List<String> sentences;
  final String remainder;
  const SentenceResult({required this.sentences, required this.remainder});
}

/// Extract completed sentences from [text] at `.`, `!`, `?` boundaries.
/// Returns the completed sentences and the unconsumed remainder.
SentenceResult extractSentences(String text) {
  final boundary = RegExp(r'[.!?]+(?=\s|$)');
  final sentences = <String>[];
  var remaining = text;
  RegExpMatch? match;
  while ((match = boundary.firstMatch(remaining)) != null) {
    final end = match!.end;
    final sentence = remaining.substring(0, end).trim();
    if (sentence.isNotEmpty) sentences.add(sentence);
    remaining = remaining.substring(end).trimLeft();
  }
  return SentenceResult(sentences: sentences, remainder: remaining);
}

/// Strip markdown formatting before sending text to TTS.
String stripMarkdownForTts(String text) {
  return text
      .replaceAll(RegExp(r'```[\s\S]*?```'), '')   // fenced code blocks
      .replaceAll(RegExp(r'`([^`]+)`'), r'$1')                // inline code — keep content
      .replaceAll(RegExp(r'#{1,6}\s+'), '')         // headings
      .replaceAll(RegExp(r'\*\*([^*]+)\*\*'), r'$1') // bold
      .replaceAll(RegExp(r'\*([^*]+)\*'), r'$1')    // italic
      .replaceAll(RegExp(r'\[([^\]]+)\]\([^)]+\)'), r'$1') // links → text
      .replaceAll(RegExp(r'^\s*[-*+]\s+', multiLine: true), '') // list markers
      .replaceAll(RegExp(r'\n{2,}'), ' ')           // multiple newlines → space
      .replaceAll('\n', ' ')
      .trim();
}

// ── State ─────────────────────────────────────────────────────────────────────

enum VoiceMode { idle, recording, transcribing, playing }

class VoiceState {
  final VoiceMode mode;
  final bool voiceModeActive;
  final bool available;

  const VoiceState({
    this.mode = VoiceMode.idle,
    this.voiceModeActive = false,
    this.available = true,
  });

  VoiceState copyWith({
    VoiceMode? mode,
    bool? voiceModeActive,
    bool? available,
  }) =>
      VoiceState(
        mode: mode ?? this.mode,
        voiceModeActive: voiceModeActive ?? this.voiceModeActive,
        available: available ?? this.available,
      );
}

// ── Provider ──────────────────────────────────────────────────────────────────

final voiceProvider =
    NotifierProvider<VoiceNotifier, VoiceState>(VoiceNotifier.new);

// ── Notifier ──────────────────────────────────────────────────────────────────

class VoiceNotifier extends Notifier<VoiceState> {
  // Audio I/O
  AudioRecorder? _recorder;
  AudioPlayer? _player;
  StreamSubscription<Amplitude>? _amplitudeSubscription;

  // Recording / silence detection
  int _recordingStartMs = 0;
  int _silenceMs = 0;
  static const _silenceThresholdDb = -40.0;
  static const _silenceDurationMs = 1500;
  static const _minRecordingMs = 300;
  String? _recordingPath;

  // Voice mode callbacks
  Future<void> Function(String transcript)? _onTranscript;
  bool _enableTts = false;

  // Streaming TTS state
  String _sentenceBuffer = '';
  int _lastSeenLength = 0;
  bool _streamComplete = false;

  // TTS playback queue
  final _ttsQueue = Queue<Uint8List>();
  bool _ttsPlaying = false;
  int _ttsCounter = 0;
  Directory? _tempDir;

  @override
  VoiceState build() {
    _recorder = AudioRecorder();
    _player = AudioPlayer();
    ref.onDispose(() {
      _amplitudeSubscription?.cancel();
      _recorder?.dispose();
      _player?.dispose();
    });
    return const VoiceState();
  }

  // ── Public API ──────────────────────────────────────────────────────────────

  /// Enter voice mode. Checks server availability and mic permission first.
  /// [onTranscript] is called with the transcript when a recording completes.
  /// [enableTts] — if true, TTS plays when [feedContent] is called.
  Future<void> enterVoiceMode({
    required Future<void> Function(String transcript) onTranscript,
    bool enableTts = false,
    required void Function(String message) onError,
  }) async {
    if (state.voiceModeActive) {
      exitVoiceMode();
      return;
    }

    // Check server availability
    try {
      final status =
          await ref.read(voiceRepositoryProvider).checkStatus();
      if (!status.fullyAvailable) {
        onError('Voice not available on this server');
        return;
      }
    } catch (_) {
      onError('Voice not available on this server');
      return;
    }

    // Check microphone permission
    final permStatus = await Permission.microphone.request();
    if (permStatus == PermissionStatus.denied ||
        permStatus == PermissionStatus.permanentlyDenied) {
      onError('Microphone permission required');
      if (permStatus == PermissionStatus.permanentlyDenied) {
        await openAppSettings();
      }
      return;
    }

    _onTranscript = onTranscript;
    _enableTts = enableTts;
    _tempDir = await getTemporaryDirectory();

    state = state.copyWith(voiceModeActive: true, available: true);
    await _startListening();
  }

  /// Exit voice mode, stop all recording and TTS.
  void exitVoiceMode() {
    _amplitudeSubscription?.cancel();
    _amplitudeSubscription = null;
    _recorder?.stop();
    _player?.stop();
    _ttsQueue.clear();
    _ttsPlaying = false;
    _sentenceBuffer = '';
    _lastSeenLength = 0;
    _streamComplete = false;
    _onTranscript = null;
    state = const VoiceState();
  }

  /// Feed streaming assistant content for TTS synthesis.
  /// Call this from screens with each updated [fullContent] string.
  /// Set [isComplete] to true when streaming has finished.
  void feedContent(String fullContent, {required bool isComplete}) {
    if (!state.voiceModeActive || !_enableTts) return;

    final delta = fullContent.length > _lastSeenLength
        ? fullContent.substring(_lastSeenLength)
        : '';
    _lastSeenLength = fullContent.length;
    _sentenceBuffer += delta;

    _dispatchSentences(flush: isComplete);

    if (isComplete) {
      _streamComplete = true;
      _checkRestartListening();
    }
  }

  // ── Internal recording ──────────────────────────────────────────────────────

  Future<void> _startListening() async {
    if (!state.voiceModeActive) return;

    _silenceMs = 0;
    _recordingStartMs = DateTime.now().millisecondsSinceEpoch;
    state = state.copyWith(mode: VoiceMode.recording);

    final dir = _tempDir ?? await getTemporaryDirectory();
    _recordingPath = '${dir.path}/voice_rec_${DateTime.now().millisecondsSinceEpoch}.webm';

    await _recorder!.start(
      const RecordConfig(encoder: AudioEncoder.opus, sampleRate: 16000),
      path: _recordingPath!,
    );

    _amplitudeSubscription?.cancel();
    _amplitudeSubscription = _recorder!
        .onAmplitudeChanged(const Duration(milliseconds: 200))
        .listen(_onAmplitude);
  }

  void _onAmplitude(Amplitude event) {
    if (!state.voiceModeActive) return;

    final elapsed =
        DateTime.now().millisecondsSinceEpoch - _recordingStartMs;
    if (elapsed < _minRecordingMs) return;

    if (event.current < _silenceThresholdDb) {
      _silenceMs += 200;
      if (_silenceMs >= _silenceDurationMs) {
        _amplitudeSubscription?.cancel();
        _handleSilence();
      }
    } else {
      _silenceMs = 0;
    }
  }

  Future<void> _handleSilence() async {
    if (!state.voiceModeActive) return;
    state = state.copyWith(mode: VoiceMode.transcribing);

    final path = await _recorder!.stop();
    if (path == null || !state.voiceModeActive) return;

    try {
      final bytes = await File(path).readAsBytes();
      await File(path).delete().catchError((_) {});

      if (!state.voiceModeActive) return;

      final transcript =
          await ref.read(voiceRepositoryProvider).transcribe(bytes);

      if (!state.voiceModeActive) return;

      if (transcript.isEmpty) {
        // Empty transcript — restart silently
        await _startListening();
        return;
      }

      // Reset TTS state for this new turn
      _sentenceBuffer = '';
      _lastSeenLength = 0;
      _streamComplete = false;

      if (_enableTts) {
        state = state.copyWith(mode: VoiceMode.playing);
      }

      await _onTranscript?.call(transcript);

      // If TTS is not enabled, loop immediately
      if (!_enableTts && state.voiceModeActive) {
        await _startListening();
      }
    } catch (_) {
      // Network/API error — exit voice mode
      exitVoiceMode();
    }
  }

  // ── Internal TTS ────────────────────────────────────────────────────────────

  void _dispatchSentences({required bool flush}) {
    final result = extractSentences(_sentenceBuffer);
    _sentenceBuffer = flush ? '' : result.remainder;

    for (final sentence in result.sentences) {
      _enqueueSentence(sentence);
    }
    if (flush && result.remainder.trim().length >= 3) {
      _enqueueSentence(result.remainder.trim());
    }
  }

  void _enqueueSentence(String sentence) {
    final stripped = stripMarkdownForTts(sentence);
    if (stripped.length < 3) return;
    // Fire-and-forget synthesis into queue
    _synthesiseSentence(stripped);
  }

  Future<void> _synthesiseSentence(String text) async {
    try {
      final wavBytes =
          await ref.read(voiceRepositoryProvider).synthesise(text);
      if (!state.voiceModeActive) return;
      _ttsQueue.add(wavBytes);
      if (!_ttsPlaying) _drainTtsQueue();
    } catch (_) {
      // Skip failed sentence — TTS errors are non-fatal
    }
  }

  Future<void> _drainTtsQueue() async {
    if (_ttsPlaying) return;
    _ttsPlaying = true;
    final dir = _tempDir ?? await getTemporaryDirectory();

    try {
      while (_ttsQueue.isNotEmpty && state.voiceModeActive) {
        final wavBytes = _ttsQueue.removeFirst();
        final path = '${dir.path}/tts_${_ttsCounter++}.wav';
        final file = File(path);
        await file.writeAsBytes(wavBytes);

        try {
          await _player!.setFilePath(path);
          await _player!.play();
          await _player!.processingStateStream.firstWhere(
            (s) => s == ProcessingState.completed || s == ProcessingState.idle,
          );
        } finally {
          await file.delete().catchError((_) {});
        }
      }
    } finally {
      _ttsPlaying = false;
    }

    _checkRestartListening();
  }

  void _checkRestartListening() {
    if (_streamComplete &&
        _ttsQueue.isEmpty &&
        !_ttsPlaying &&
        state.voiceModeActive) {
      _streamComplete = false;
      _lastSeenLength = 0;
      _sentenceBuffer = '';
      _startListening();
    }
  }
}

• Step 4: Run tests to confirm they pass

flutter test test/widget_test.dart

Expected: all tests pass.

• Step 5: Verify no analyzer errors

flutter analyze

Expected: no errors.

• Step 6: Commit

git add lib/providers/voice_provider.dart test/widget_test.dart
git commit -m "feat(voice): add VoiceNotifier with recording, silence detection, and streaming TTS"

---

Task 5: VoiceMicButton widget

Files:

• Create: lib/widgets/voice_mic_button.dart

• Step 1: Create lib/widgets/voice_mic_button.dart

import 'package:flutter/material.dart';

import '../providers/voice_provider.dart';

/// Animated mic button that reflects the current [VoiceMode].
///
/// - idle: muted background, mic_none icon
/// - recording: red with pulsing shadow ring
/// - transcribing: indigo with spinner
/// - playing: indigo with volume_up icon
class VoiceMicButton extends StatefulWidget {
  final VoiceMode mode;
  final bool voiceModeActive;
  final VoidCallback? onTap;

  const VoiceMicButton({
    super.key,
    required this.mode,
    required this.voiceModeActive,
    this.onTap,
  });

  @override
  State<VoiceMicButton> createState() => _VoiceMicButtonState();
}

class _VoiceMicButtonState extends State<VoiceMicButton>
    with SingleTickerProviderStateMixin {
  late AnimationController _pulseController;
  late Animation<double> _pulseAnimation;

  @override
  void initState() {
    super.initState();
    _pulseController = AnimationController(
      vsync: this,
      duration: const Duration(milliseconds: 900),
    )..repeat(reverse: true);
    _pulseAnimation = Tween<double>(begin: 1.0, end: 1.25).animate(
      CurvedAnimation(parent: _pulseController, curve: Curves.easeInOut),
    );
  }

  @override
  void dispose() {
    _pulseController.dispose();
    super.dispose();
  }

  Color _bgColor(BuildContext context) {
    final cs = Theme.of(context).colorScheme;
    return switch (widget.mode) {
      VoiceMode.recording => const Color(0xFFEF4444),
      VoiceMode.transcribing || VoiceMode.playing => cs.primary,
      VoiceMode.idle => cs.surfaceContainerHighest,
    };
  }

  Widget _icon(BuildContext context) {
    final cs = Theme.of(context).colorScheme;
    final iconColor = widget.mode == VoiceMode.idle
        ? cs.onSurfaceVariant
        : Colors.white;

    return switch (widget.mode) {
      VoiceMode.transcribing => SizedBox(
          width: 18,
          height: 18,
          child: CircularProgressIndicator(
            strokeWidth: 2,
            color: iconColor,
          ),
        ),
      VoiceMode.playing => Icon(Icons.volume_up, color: iconColor, size: 20),
      _ => Icon(Icons.mic, color: iconColor, size: 20),
    };
  }

  @override
  Widget build(BuildContext context) {
    final isRecording = widget.mode == VoiceMode.recording;

    final button = Material(
      color: _bgColor(context),
      shape: const CircleBorder(),
      child: InkWell(
        customBorder: const CircleBorder(),
        onTap: widget.onTap,
        child: SizedBox(
          width: 40,
          height: 40,
          child: Center(child: _icon(context)),
        ),
      ),
    );

    if (!isRecording) return button;

    return AnimatedBuilder(
      animation: _pulseAnimation,
      builder: (_, child) => Container(
        decoration: BoxDecoration(
          shape: BoxShape.circle,
          boxShadow: [
            BoxShadow(
              color: const Color(0xFFEF4444).withValues(alpha: 0.35),
              blurRadius: 8 * _pulseAnimation.value,
              spreadRadius: 2 * _pulseAnimation.value,
            ),
          ],
        ),
        child: child,
      ),
      child: button,
    );
  }
}

• Step 2: Verify no errors

flutter analyze

Expected: no errors.

• Step 3: Commit

git add lib/widgets/voice_mic_button.dart
git commit -m "feat(voice): add VoiceMicButton widget with animated states"

---

Task 6: Chat screen integration

Files:

• Modify: lib/screens/chat/chat_screen.dart

The Chat screen needs:

1. A VoiceMicButton next to the send button
2. A red banner above the input row when voice mode is active
3. A ref.listen on messagesProvider that feeds content to VoiceNotifier during TTS

• Step 1: Add voice imports and mic button to chat_screen.dart

Replace the full contents of lib/screens/chat/chat_screen.dart with:

import 'package:flutter/material.dart';
import 'package:flutter_riverpod/flutter_riverpod.dart';

import '../../core/exceptions.dart';
import '../../data/models/message.dart';
import '../../providers/chat_provider.dart';
import '../../providers/voice_provider.dart';
import '../../widgets/chat_message_bubble.dart';
import '../../widgets/voice_mic_button.dart';

class ChatScreen extends ConsumerStatefulWidget {
  final int conversationId;
  const ChatScreen({super.key, required this.conversationId});

  @override
  ConsumerState<ChatScreen> createState() => _ChatScreenState();
}

class _ChatScreenState extends ConsumerState<ChatScreen> {
  final _controller = TextEditingController();
  final _scrollController = ScrollController();

  @override
  void dispose() {
    _controller.dispose();
    _scrollController.dispose();
    // Exit voice mode if the user navigates away mid-session.
    ref.read(voiceProvider.notifier).exitVoiceMode();
    super.dispose();
  }

  void _scrollToBottom() {
    WidgetsBinding.instance.addPostFrameCallback((_) {
      if (_scrollController.hasClients) {
        _scrollController.animateTo(
          _scrollController.position.maxScrollExtent,
          duration: const Duration(milliseconds: 200),
          curve: Curves.easeOut,
        );
      }
    });
  }

  Future<void> _send() async {
    final text = _controller.text.trim();
    if (text.isEmpty) return;
    _controller.clear();
    try {
      await ref
          .read(messagesProvider(widget.conversationId).notifier)
          .sendMessage(text);
    } on AppException catch (e) {
      if (mounted) {
        ScaffoldMessenger.of(context)
            .showSnackBar(SnackBar(content: Text(e.message)));
      }
    } catch (_) {
      if (mounted) {
        ScaffoldMessenger.of(context).showSnackBar(
          const SnackBar(content: Text('Failed to send message.')),
        );
      }
    }
  }

  Future<void> _toggleVoiceMode() async {
    final voice = ref.read(voiceProvider);
    if (voice.voiceModeActive) {
      ref.read(voiceProvider.notifier).exitVoiceMode();
      return;
    }
    await ref.read(voiceProvider.notifier).enterVoiceMode(
      onTranscript: (transcript) async {
        await ref
            .read(messagesProvider(widget.conversationId).notifier)
            .sendMessage(transcript);
      },
      enableTts: true,
      onError: (msg) {
        if (mounted) {
          ScaffoldMessenger.of(context)
              .showSnackBar(SnackBar(content: Text(msg)));
        }
      },
    );
  }

  @override
  Widget build(BuildContext context) {
    final messagesAsync = ref.watch(messagesProvider(widget.conversationId));
    final isStreaming = ref.watch(isStreamingProvider(widget.conversationId));
    final voiceState = ref.watch(voiceProvider);

    // Scroll when messages change.
    ref.listen(messagesProvider(widget.conversationId), (_, _) {
      _scrollToBottom();
    });

    // Feed streaming content to VoiceNotifier for TTS.
    ref.listen(messagesProvider(widget.conversationId), (_, next) {
      if (!voiceState.voiceModeActive) return;
      final messages = next.value;
      if (messages == null || messages.isEmpty) return;
      final last = messages.last;
      if (last.role != MessageRole.assistant) return;
      final isComplete = last.status != 'generating';
      ref
          .read(voiceProvider.notifier)
          .feedContent(last.content, isComplete: isComplete);
    });

    final convTitle = ref
        .watch(conversationsProvider)
        .value
        ?.where((c) => c.id == widget.conversationId)
        .firstOrNull
        ?.title;

    return Scaffold(
      appBar: AppBar(
        title: Text(convTitle?.isNotEmpty == true ? convTitle! : 'Chat'),
      ),
      body: Column(
        children: [
          Expanded(
            child: messagesAsync.when(
              loading: () =>
                  const Center(child: CircularProgressIndicator()),
              error: (_, _) => const Center(
                child: Text('Could not load messages.'),
              ),
              data: (messages) {
                if (messages.isEmpty) {
                  return const Center(
                      child: Text('Send a message to start.'));
                }
                return ListView.builder(
                  controller: _scrollController,
                  padding: const EdgeInsets.symmetric(
                      horizontal: 8, vertical: 12),
                  itemCount: messages.length,
                  itemBuilder: (context, i) =>
                      ChatMessageBubble(message: messages[i]),
                );
              },
            ),
          ),
          // Voice mode banner
          if (voiceState.voiceModeActive)
            Container(
              width: double.infinity,
              color: const Color(0xFFEF4444).withValues(alpha: 0.12),
              padding:
                  const EdgeInsets.symmetric(horizontal: 16, vertical: 6),
              child: const Text(
                '🎤 Listening… tap mic to exit voice mode',
                style: TextStyle(
                  fontSize: 12,
                  color: Color(0xFFF87171),
                ),
              ),
            ),
          const Divider(height: 1),
          SafeArea(
            child: Padding(
              padding:
                  const EdgeInsets.symmetric(horizontal: 8, vertical: 6),
              child: Row(
                children: [
                  Expanded(
                    child: TextField(
                      controller: _controller,
                      decoration: InputDecoration(
                        hintText: voiceState.voiceModeActive
                            ? 'Listening…'
                            : 'Message…',
                        hintStyle: voiceState.voiceModeActive
                            ? const TextStyle(fontStyle: FontStyle.italic)
                            : null,
                        border: const OutlineInputBorder(),
                        isDense: true,
                        contentPadding: const EdgeInsets.symmetric(
                            horizontal: 12, vertical: 10),
                      ),
                      maxLines:
                          MediaQuery.of(context).size.width >= 600 ? 2 : 4,
                      minLines: 1,
                      textInputAction: TextInputAction.newline,
                      enabled: !isStreaming && !voiceState.voiceModeActive,
                    ),
                  ),
                  const SizedBox(width: 8),
                  VoiceMicButton(
                    mode: voiceState.mode,
                    voiceModeActive: voiceState.voiceModeActive,
                    onTap: _toggleVoiceMode,
                  ),
                  const SizedBox(width: 6),
                  IconButton.filled(
                    onPressed: (isStreaming || voiceState.voiceModeActive)
                        ? null
                        : _send,
                    icon: isStreaming
                        ? const SizedBox(
                            width: 20,
                            height: 20,
                            child:
                                CircularProgressIndicator(strokeWidth: 2),
                          )
                        : const Icon(Icons.send),
                  ),
                ],
              ),
            ),
          ),
        ],
      ),
    );
  }
}

• Step 2: Verify no errors

flutter analyze

Expected: no errors.

• Step 3: Commit

git add lib/screens/chat/chat_screen.dart
git commit -m "feat(voice): integrate voice mode into Chat screen"

---

Task 7: Quick Capture bar integration

Files:

• Modify: lib/app.dart

The _QuickCaptureBarState needs a VoiceMicButton. Recording is one-shot: transcript goes to captureWorkQueueProvider.enqueue(), no TTS, no loop. The mic button sits between the text field and the settings gear icon.

• Step 1: Add imports to lib/app.dart

At the top of lib/app.dart, add after the existing widget imports:

import 'providers/voice_provider.dart';
import 'widgets/voice_mic_button.dart';

• Step 2: Add _toggleCaptureMic method to _QuickCaptureBarState

In _QuickCaptureBarState, add this method after _drainOfflineQueue:

  Future<void> _toggleCaptureMic() async {
    final voice = ref.read(voiceProvider);
    if (voice.voiceModeActive) {
      ref.read(voiceProvider.notifier).exitVoiceMode();
      return;
    }
    await ref.read(voiceProvider.notifier).enterVoiceMode(
      onTranscript: (transcript) async {
        ref.read(captureWorkQueueProvider.notifier).enqueue(transcript);
      },
      enableTts: false,
      onError: (msg) {
        if (!mounted) return;
        ScaffoldMessenger.of(context)
            .showSnackBar(SnackBar(content: Text(msg)));
      },
    );
  }

• Step 3: Add VoiceMicButton to the capture bar Row

In _QuickCaptureBarState.build(), locate the Row inside the Padding widget. The existing row has: [Expanded(TextField), IconButton(settings)].

Replace that Row's children list with:

                children: [
                  Expanded(
                    child: TextField(
                      controller: _controller,
                      textInputAction: TextInputAction.send,
                      onSubmitted: (_) => _submit(),
                      onChanged: (_) => setState(() {}),
                      decoration: InputDecoration(
                        hintText: ref.watch(voiceProvider).voiceModeActive
                            ? 'Listening…'
                            : _hintForLocation(location),
                        isDense: true,
                        contentPadding: const EdgeInsets.symmetric(
                            horizontal: 14, vertical: 10),
                        prefixIcon: totalPending > 0
                            ? Badge(
                                label: Text('$totalPending'),
                                child: const Icon(Icons.cloud_upload_outlined),
                              )
                            : isWorking
                                ? const Padding(
                                    padding: EdgeInsets.all(12),
                                    child: SizedBox(
                                      width: 16,
                                      height: 16,
                                      child: CircularProgressIndicator(
                                          strokeWidth: 2),
                                    ),
                                  )
                                : const Icon(Icons.auto_awesome_outlined),
                        suffixIcon: _controller.text.trim().isNotEmpty
                            ? IconButton(
                                icon: const Icon(Icons.send),
                                onPressed: _submit,
                                tooltip: 'Capture',
                              )
                            : null,
                      ),
                    ),
                  ),
                  VoiceMicButton(
                    mode: ref.watch(voiceProvider).mode,
                    voiceModeActive: ref.watch(voiceProvider).voiceModeActive,
                    onTap: _toggleCaptureMic,
                  ),
                  IconButton(
                    icon: const Icon(Icons.settings_outlined),
                    tooltip: 'Settings',
                    onPressed: () => context.push(Routes.settings),
                  ),
                ],

• Step 4: Verify no errors

flutter analyze

Expected: no errors.

• Step 5: Commit

git add lib/app.dart
git commit -m "feat(voice): integrate one-shot voice capture into Quick Capture bar"

---

Task 8: Briefing screen integration

Files:

• Modify: lib/screens/briefing/briefing_screen.dart

Same pattern as Chat: mic button in the reply row, voice banner above input, and ref.listen on briefingProvider to feed TTS.

• Step 1: Add imports to briefing_screen.dart

At the top of lib/screens/briefing/briefing_screen.dart, add after existing imports:

import '../../data/models/message.dart';
import '../../providers/voice_provider.dart';
import '../../widgets/voice_mic_button.dart';

• Step 2: Add exitVoiceMode to dispose

In _BriefingScreenState.dispose(), add before super.dispose():

    ref.read(voiceProvider.notifier).exitVoiceMode();

• Step 3: Add _toggleVoiceMode method

In _BriefingScreenState, add after _refresh:

  Future<void> _toggleVoiceMode() async {
    final voice = ref.read(voiceProvider);
    if (voice.voiceModeActive) {
      ref.read(voiceProvider.notifier).exitVoiceMode();
      return;
    }
    await ref.read(voiceProvider.notifier).enterVoiceMode(
      onTranscript: (transcript) async {
        await ref.read(briefingProvider.notifier).sendReply(transcript);
      },
      enableTts: true,
      onError: (msg) {
        if (mounted) {
          ScaffoldMessenger.of(context)
              .showSnackBar(SnackBar(content: Text(msg)));
        }
      },
    );
  }

• Step 4: Add TTS feed listener in build

In _BriefingScreenState.build(), locate the existing ref.listen(briefingProvider, ...) call that scrolls to bottom. Add a second ref.listen immediately after it:

    // Feed streaming assistant content to VoiceNotifier for TTS.
    ref.listen(briefingProvider, (_, next) {
      final voiceState = ref.read(voiceProvider);
      if (!voiceState.voiceModeActive) return;
      final conv = next.value;
      if (conv == null || conv.messages.isEmpty) return;
      final last = conv.messages.last;
      if (last.role != MessageRole.assistant) return;
      final isComplete = last.status != 'generating';
      ref
          .read(voiceProvider.notifier)
          .feedContent(last.content, isComplete: isComplete);
    });

• Step 5: Add voice banner and mic button to the reply row

Locate the reply bar section in build() — the Column inside data: (conv). It currently ends with:

              // Reply bar
              const Divider(height: 1),
              SafeArea(
                child: Padding(
                  padding: const EdgeInsets.fromLTRB(8, 6, 8, 6),
                  child: Row(
                    children: [
                      Expanded(
                        child: TextField( ... ),
                      ),
                      const SizedBox(width: 8),
                      _GradientSendButton(
                        onPressed: isStreaming ? null : _sendReply,
                        isStreaming: isStreaming,
                      ),

Replace that section (from // Reply bar to the closing of its Column entry) with:

              // Voice mode banner
              if (voiceState.voiceModeActive)
                Container(
                  width: double.infinity,
                  color: const Color(0xFFEF4444).withValues(alpha: 0.12),
                  padding: const EdgeInsets.symmetric(
                      horizontal: 16, vertical: 6),
                  child: const Text(
                    '🎤 Listening… tap mic to exit voice mode',
                    style: TextStyle(
                      fontSize: 12,
                      color: Color(0xFFF87171),
                    ),
                  ),
                ),
              // Reply bar
              const Divider(height: 1),
              SafeArea(
                child: Padding(
                  padding: const EdgeInsets.fromLTRB(8, 6, 8, 6),
                  child: Row(
                    children: [
                      Expanded(
                        child: TextField(
                          controller: _controller,
                          decoration: InputDecoration(
                            hintText: voiceState.voiceModeActive
                                ? 'Listening…'
                                : 'Reply to your briefing…',
                            hintStyle: voiceState.voiceModeActive
                                ? const TextStyle(fontStyle: FontStyle.italic)
                                : null,
                            border: const OutlineInputBorder(),
                            isDense: true,
                            contentPadding: const EdgeInsets.symmetric(
                                horizontal: 12, vertical: 10),
                          ),
                          minLines: 1,
                          maxLines: 4,
                          textInputAction: TextInputAction.newline,
                          enabled: !isStreaming && !voiceState.voiceModeActive,
                        ),
                      ),
                      const SizedBox(width: 8),
                      VoiceMicButton(
                        mode: voiceState.mode,
                        voiceModeActive: voiceState.voiceModeActive,
                        onTap: _toggleVoiceMode,
                      ),
                      const SizedBox(width: 6),
                      _GradientSendButton(
                        onPressed: (isStreaming || voiceState.voiceModeActive)
                            ? null
                            : _sendReply,
                        isStreaming: isStreaming,
                      ),

• Step 6: Add voiceState variable in build

At the top of _BriefingScreenState.build(), just after the existing final isStreaming = ... line, add:

    final voiceState = ref.watch(voiceProvider);

• Step 7: Verify no errors

flutter analyze

Expected: no errors.

• Step 8: Commit

git add lib/screens/briefing/briefing_screen.dart
git commit -m "feat(voice): integrate voice mode into Briefing screen"

---

Task 9: Final verification

• Step 1: Run all tests

cd /home/bvandeusen/Nextcloud/Projects/fabled_app
flutter test test/widget_test.dart

Expected: all tests pass.

• Step 2: Full static analysis

flutter analyze

Expected: no errors or warnings related to voice code.

• Step 3: Build debug APK to confirm it compiles

flutter build apk --debug 2>&1 | tail -20

Expected: ✓ Built build/app/outputs/flutter-apk/app-debug.apk

• Step 4: Push to dev

cd /home/bvandeusen/Nextcloud/Projects/fabled_app
git push origin dev

---

Manual smoke test checklist

After installing the debug APK on device:

• Chat screen: tap mic → permission dialog appears on first use
• Chat screen: speak → silence → transcript appears and message sends
• Chat screen: assistant response audio plays sentence by sentence
• Chat screen: tap mic again → voice mode exits, input re-enables
• Chat screen: navigate away mid-session → no audio or recording continues
• Quick Capture: tap mic → speak → transcript captured as note/task, mic returns to idle
• Briefing screen: tap mic → speak → reply sent, TTS plays response
• Server with voice disabled: tap mic → snackbar "Voice not available on this server"
• Deny mic permission: snackbar "Microphone permission required"