The Voice Agent API provides real-time spoken conversation over one WebSocket connection. Stream microphone audio to an agent and receive live transcripts, text, synthesized speech, and function calls.
Use the Voice Agent API for customer support, scheduling assistants, voice interfaces, language practice, kiosks, and ordering systems.
Key features
- Bidirectional audio streaming over WebSocket
- Automatic or client-controlled turn detection
- Barge-in, so a user can interrupt a response in progress
- Transcription for user and agent speech
- Streamed audio or text responses
- Synthesized speech at 16 kHz with ten available voices
- Function calling for integrations with your application
- Managed conversation history, trimmed automatically by default as it approaches the model's context limit
Concepts
| Concept | Description |
|---|---|
| Session | The active model, voice, instructions, turn detection, tools, and audio configuration. |
| Conversation | Ordered history accumulated during the WebSocket session. |
| Item | A user or agent message, function call, or function result in the conversation. |
| Response | One agent turn, including its streamed output, status, and usage. |
How it works
- Connect to the Voice Agent WebSocket endpoint.
- Receive
session.createdand sendsession.updateas the first client event. - Stream Base64-encoded input audio in JSON events.
- Handle turn, transcription, response, and audio events.
- Execute requested functions and return their results to the conversation.
For protocol details and examples, see Live sessions.
Resources
AI Services are part of the Zoom Build platform. Use Zoom Platform Studio to create a project, get credentials, and manage your AI Services plan.