Zoom Scribe API: Introducing Live mode

We launched Zoom AI Services with the Scribe API for accurate, scalable transcription. Scribe already supports synchronous transcription for individual files through Fast mode and asynchronous processing for large collections through Batch mode.

Today, we're adding Live mode: a real-time transcription API for voice agents and interactive audio applications.

Transcription as the conversation happens

File-based transcription works well when audio is already recorded. Interactive applications need text while someone is still speaking.

Live mode opens a secure WebSocket connection between your backend and Scribe. Your application streams audio as binary messages, and Scribe returns a transcript segment after each detected speech turn.

This event-driven workflow supports experiences such as:

• Voice agents that respond without waiting for a recording to finish
• Real-time keyword detection and agent assistance
• Conversational interfaces that act on finalized speech turns

One API, three processing modes

Live mode joins Fast and Batch to give developers a processing option for each stage of an audio workflow.

Applications can combine these modes. For example, use Live mode to show a transcript during a customer conversation, then use Batch mode to process stored recordings for quality assurance and analytics.

How Live mode works

Your backend connects to the Scribe Live WebSocket with a Zoom AI Services JWT and the live-asr WebSocket subprotocol. After the connection opens:

1. Send a session.update message with the language and audio format.
2. Stream 16-bit, 16 kHz, mono PCM audio as binary WebSocket messages.
3. Handle speech and transcription events returned by Scribe.
4. Send session.close when transcription is complete.

The session configuration defines the language and audio format:

{
    "type": "session.update",
    "language": "en-US",
    "audio": {
        "format": "pcm16"
    }
}

Scribe sends lifecycle events such as input_audio_buffer.speech_started and input_audio_buffer.speech_stopped, followed by the text for a completed speech turn through transcription.completed.

For browser microphone capture, connect the browser to your backend through a WebSocket proxy. Browsers cannot add an Authorization header to a WebSocket connection, and keeping the JWT on your backend prevents it from being exposed in client-side code.

Get started

Zoom Scribe Live mode is available through the Zoom Build platform. Read the Live mode documentation to configure a session, stream audio, handle transcription events, and close the connection. You can try out Live mode with the AI Service Quickstart repository on Github.

With Live, Fast, and Batch modes, Scribe can now support transcription from the first spoken word through large-scale processing of your audio archive.