KeyanVakil/add-voice-ai

Give the app a voice.

Steps

1. Determine the voice use case:

   • Outbound calls (app calls user) → Bland AI
   • Inbound calls (user calls a number) → Bland AI
   • In-browser TTS (speak responses aloud) → ElevenLabs or Web Speech API
   • In-browser STT (transcribe user's mic) → Deepgram or Web Speech API
   • Full voice agent (speak + listen in browser) → Deepgram + ElevenLabs or Bland AI web

2. Implement based on use case:

Bland AI (calls)

POST https://api.bland.ai/v1/calls
Authorization: Bearer $BLAND_API_KEY
{
  "phone_number": "+1...",
  "task": "...",
  "voice": "...",
  "model": "enhanced",
  "max_duration": 12
}

• Use pathway_id for multi-step call flows
• Handle webhooks at /api/bland/webhook for call events and transcripts
• Store call_id to retrieve recordings/transcripts later

ElevenLabs TTS (browser)

• Install elevenlabs SDK
• Stream audio with textToSpeech.stream() → pipe to <audio> element
• Use a server route to proxy API calls — never expose ELEVENLABS_API_KEY to the client

Deepgram STT (browser)

• Use @deepgram/sdk with createClient().listen.live()
• Request mic permission, stream getUserMedia audio chunks to Deepgram WebSocket
• Emit transcript events to your app state

Web Speech API (no API key, limited browser support)

• SpeechSynthesis for TTS, SpeechRecognition for STT
• Works in Chrome/Edge only, no Safari — add a fallback warning

3. Add required env vars to .env.example.
4. If adding calls: build a simple call log UI showing status, duration, transcript.

Rules

• Never stream API keys to the browser — always proxy through your own server route
• Add rate limiting to any route that triggers an outbound call (prevent accidental spam)
• Store call IDs and transcripts — they're useful for debugging and user history
• Test with short max_duration values during development