Use raw data

Access raw video and raw audio data through raw recording or raw streaming. When accessing raw data through raw recording, the host gives the meeting participant associated with the SDK app local recording permission. When accessing raw data through raw streaming, the host gives the meeting participant associated with the SDK app live streaming permission.

This enables more flexibility than either local recording or live streaming for video and audio streams. In addition to session-wide audio and video access, this enables access to raw audio and video data for each participant unless the participant's audio or video are muted.

For raw recording, if the SDK user is also the meeting host, they can start both raw recording and an actual local recording. If the SDK user is not the meeting host, they must request local recording permission from the meeting host. In a meeting, the host can perform cloud recording, with one or more SDK clients doing raw recording locally.

For raw streaming, if the SDK user is also the meeting host, they can start both raw streaming and an actual live streaming service. If the SDK user is not the meeting host, they must request raw streaming permission from the meeting host. In a meeting, the host can perform real external live streaming, with one or more SDK clients doing raw streaming locally.

The SDKs prompt users with a dialog to let them know when recording or streaming has started, as in the Zoom clients used by other meeting participants.

Raw data formats#

• Video is in YUV420p format
• Audio is in PCM format

Before you start#

To work with raw data in Meeting SDK for macOS, you must:

• Have Meeting SDK for macOS version 5.9.0 or later
  • Raw streaming by meeting host: Meeting SDK for macOS version 5.11.0 or later
  • Raw streaming by non-host participant: Meeting SDK for macOS version 5.12.8 or later
  • To request local recording permission from host: Meeting SDK for Windows version 5.13.5 or later

• Meet these requirements for using raw recording
  • The meeting must enable local recording 
  • Meet any one of these additional requirements
    • You are a normal participant but with local recording permission 
    • You are the meeting host or co-host

• Meet all of these requirements for raw streaming
  • The current user has raw live streaming permission
  • The meeting host has a Pro, Business, Education, or Enterprise account
  • The meeting host is licensed 

Recording and streaming sequence diagrams#

This sequence diagram shows the raw recording flow.

This sequence diagram shows the raw streaming flow.

Usage#

• Set up video raw data callbacks
• Set up audio raw data callbacks
• Request LocalRecording permission
• Call StartRawRecording to start recording raw data
• Call StopRawRecording to stop recording raw data
  OR
• Request RawLiveStream permission
• Call StartRawLiveStream to start streaming raw data
• Call StopRawLiveStream to stop streaming raw data

Methods#

• Video raw data callbacks : onRawDataFrameReceived — triggered for each video frame
• Audio raw data callbacks 
  • onOneWayAudioRawDataReceived — per-user audio (no support for telephone participants)
  • onMixedAudioRawDataReceived — session-wide audio (what meeting participants hear)
• startRawRecording  — start raw recording
• stopRawRecording  — stop raw recording
• StartRawLiveStream  — start raw streaming
• StopRawLiveStream  — stop raw streaming

Record raw data#

Follow these steps to record audio, video, or a screen.

1. Implement an instance of ZoomSDKRendererDelegate.

   • In this class, respond to onRawDataReceived to process the raw video or raw screen data stored in the ZoomSDKYUVRawDataI420 object.
   • Similarly, set up an ZoomSDKAudioRawDataDelegate for the audio raw data.
   • There are two different callbacks that can be used to receive raw data: onOneWayAudioRawDataReceived for per-user audio, and onMixedAudioRawDataReceived for meeting-wide audio.

   Note: Do not perform heavy operations from within the raw data callbacks. Doing so may cause delayed or lost data.

2. If the current user has permission to start the recording, call startRawRecording in ZoomSDKMeetingRecordController to trigger the recording consent dialog for all users.

   • A user can either request local recording permission by asking the meeting host, or by joining the meeting with a special local recording token (see instructions below.)
   • A raw recording is a type of local recording, so listen for the onRecordStatus  callback to determine when the recording successfully starts.

3. Once the recording starts, set up a ZoomSDKRenderer, set your ZoomSDKRendererDelegate as the delegate property, and subscribe to the user via the subscribe method.

   • For audio, set your ZoomSDKAudioRawDataDelegate as the delegate property, but you don't need to provide a userId as all callbacks will go through the same ZoomSDKAudioRawDataHelper instance.

4. To end the recording, call stopRawRecording in ZoomSDKMeetingRecordController.

   • Unsubscribe in any ZoomSDKRenderer and ZoomSDKAudioRawDataHelper instance(s) you were using.

Receive local recording permission through an OAuth app#

To successfully request recording permission, your OAuth app must be:

• a public OAuth app in the Zoom Marketplace
• installed by the meeting host or by their account admin

1. Your OAuth app requests one of the two scopes:
   • Meeting_token:read:admin:local_recording - account level, installed by admin
   • Meeting_token:read:local_recording - user level, installed by host
2. Once the SDK app has the meeting ID it needs to join, have it call the "Get a meeting's join token for local recording" REST API endpoint
3. Have your OAuth app pass the returned token to your Meeting SDK app.
4. Have the Meeting SDK app use the returned token to join the meeting with the app_privilege_token  parameter.

Stream raw data from within the current meeting#

Follow these steps to stream either audio or video from inside a meeting.

1. Implement an instance of ZoomSDKRendererDelegate.

   • In this class, respond to onRawDataReceived to process the raw video data stored in the YUVRawDataI420 object.
   • Additionally, set up an ZoomSDKAudioRawDataDelegate for the raw audio data.
   • Use one of two different callbacks to receive raw data: onOneWayAudioRawDataReceived for per-user audio, or onMixedAudioRawDataReceived for meeting-wide audio.

   Note: Do not perform heavy operations from within the raw data callbacks. This may cause delayed or lost data.

2. Call StartRawLiveStream in ZoomSDKLiveStreamHelper to trigger the live streaming consent dialog for all users if the current user has permission to start the live streaming.

   • A raw stream is a type of live streaming, so listen for the onUserRawLiveStreamingStatusChanged  callback to determine when the raw live streaming successfully starts.

3. Once the streaming starts, pass the ZoomRendererDelegate into the ZoomSDKRenderer.delegate property.

   • For audio, pass your delegate into the ZoomSDKAudioRawDataHelper class subscribe function.

4. To end the recording, call StopRawLiveStream in MeetingLiveStreamHelper.

   • Unsubscribe in any ZoomSDKRenderer or ZoomSDKRawDataHelper instances you used.

Notify users when beginning a live stream#

When beginning a live stream inside a current meeting, the Meeting SDK displays a notice telling users that the following portion of the meeting will be streamed to another platform. This gives users an informed chance to opt out of being streamed on that new platform.

• RawLiveStreamInfo.broadcastName - the name of the app opening the new streaming session, such as "AI Note taker by XYZ"
• RawLiveStreamInfo.broadcastUrl - URL of a landing page hosted by the app developer, OR the URL of the app's Zoom Marketplace page, such as https://example.com/description-page or https://marketplace.zoom.us/apps/XXXXXXexampleXXXX.

Request live streaming permission#

If you do not have live streaming permission, you can either request this permission from the meeting host during the meeting or request live-streaming permission through the OAuth app, so that the SDK client can start and stop raw streaming.

Request live streaming permission from the host#

To request live streaming permission from host during the meeting:

1. Call RequestRawLiveStream() .
2. Receive permission by checking callback onRawLiveStreamPrivilegeChanged() .

Request live-streaming permission through an OAuth app#

To successfully request live-streaming permission through an OAuth app, your app must be:

• a public OAuth app on the Zoom Marketplace
• installed by the meeting host or by their account admin

1. Your OAuth app will request one of the two scopes:
   • Meeting_token:read:admin:live_streaming - account level, installed by admin
   • Meeting_token:read:live_streaming - user level, installed by host
2. Once your SDK app has the meeting ID it needs to join, it calls the "Get a meeting's join token for live streaming" REST API endpoint
3. Have the OAuth app pass the returned token to your Meeting SDK app.
4. Have the Meeting SDK app use the returned token to join the meeting with the app_privilege_token  parameter.

Share raw data#

Follow these steps to share audio, video, or a screen.

Share audio in PCM buffer format#

1. Implement an instance of ZoomSDKVirtualAudioMicDelegate.
   • Save a reference to the method parameter ZoomSDKAudioRawDataSender pSender in the onMicInitialize method.
   • Respond to onMicStartSend. Using your reference to pSender, call [pSender send:dataBuffer dataLength:dataLength sampleRate:sampleRate]; .
2. Call getAudioRawDataHelper, which returns the ZoomSDKAudioRawDataHelper object.
3. Pass the implementation of ZoomSDKAudioRawDataDelegate into the setExternalAudioSource method found in the ZoomSDKAudioRawDataHelper object.

onMicStartSend is invoked. You can now send audio in PCM buffer format.

Share video in YUV420 buffer format#

1. Implement an instance of ZoomSDKVirtualVideoSourceDelegate.
   • Save a reference to the method parameter ZoomSDKRawDataSender sender found in the onInitialize method.
   • Respond to onStartSend. Using your reference to sender, to call [sender sendRawData:dataBuffer width:width height:height dataLength:dataLength ratation:rotation];.
2. Call getRawDataVideoSourceHelper, which returns the ZoomSDKRawDataVideoSourceController object.
3. Pass the implementation of ZoomSDKVirtualVideoSourceDelegate into the setExternalVideoSource method, found in the ZoomSDKRawDataVideoSourceController object.

onStartSend is invoked. You can now send video in YUV420 buffer format.

Share a screen in YUV420#

1. Implement an instance of ZoomSDKShareSourceDelegate
   • Respond to onStartSend to send video in YUV420 buffer format. Use (ZoomSDKShareSender *) sender in the method parameter to call [sender sendShareFrame:frameBuffer width:width height:height frameLength:frameLength];.
2. Call getRawDataShareSourceHelper, which returns the ZoomSDKRawDataShareSourceController object.
3. Pass the implementation of ZoomSDKShareSourceDelegate into the setExternalShareSource method, found in the ZoomSDKRawDataShareSourceController object.

onStartSend is invoked. You can now screen-share in YUV420 buffer format.

Notes#

• The usage or actual handling of the raw data may vary based on your use case.
• The SDK only provides the raw data. Implement only the raw data video or audio callbacks that you need.
• Performing any heavy handling within the callback delays the callback being triggered, which may result in loss of data.