Use raw data

The code on this page works with either the default UI or the custom UI.

The Meeting SDK for Linux provides you with an option to access real-time raw audio and video of a meeting. After accessing the raw data, you can process it and apply additional effects to enhance in-meeting or post-meeting experience. This section describes how to enable and gain access to a session's raw data.

Prerequisites

The Meeting SDK for Linux requires pulseaudio to enable audio raw data function.

If you're using a Docker container to create your app, install pulseaudio before implementing the audio raw data function, then create a configuration file at ~/.config/zoomus.conf and add this content.

system.audio.type=default

Select raw data memory mode

You must select a memory mode to receive any type of raw data. The Meeting SDK supports heap-based and stack-based memory modes.

Compare stack-based and heap-based memory

Specify memory mode

After determining which memory mode is right for you, specify the memory mode when initializing the SDK. This must be done for audio, video, and share data types individually. To specify a raw data memory mode, provide one of these enums cases to the InitParam during SDK initialization. The two types of memory allocation are defined with these enum values.

InitParam init_params;
init_params.rawdataOpts.videoRawdataMemoryMode = ZoomSDKRawDataMemoryModeHeap;
init_params.rawdataOpts.shareRawDataMemoryMode = ZoomSDKRawDataMemoryModeHeap;
init_params.rawdataOpts.audioRawDataMemoryMode = ZoomSDKRawDataMemoryModeHeap;

Receive raw video data

Raw video data is encoded in the YUV420p format. YUV420 is a data object commonly used by the renderer based on OpenGL ES. To access and modify the video data, follow these steps.

1. Implement an instance of the IZoomSDKRendererDelegate.
2. Use callback functions provided by the IZoomSDKRendererDelegate to receive each frame of the raw video data.
3. Pass the userID to IZoomSDKRenderer's subscribe function.

ZoomSDKRendererDelegate* videoSource = new ZoomSDKRendererDelegate();
IZoomSDKRenderer* videoHelper;
SDKError err1 = m_pRecordController->StartRawRecording();
if (err1 != SDKERR_SUCCESS) {
	std::cout << "Error occurred" << std::endl;
}
SDKError err = createRenderer(&videoHelper, videoSource);
if (err != SDKERR_SUCCESS) {
	std::cout << "Error occurred" << std::endl;
	//handle error
}
else {
	std::cout << "attemptToStartRawRecording : subscribing" << std::endl;
	videoHelper->setRawDataResolution(ZoomSDKResolution_720P);
	videoHelper->subscribe(getUserID(), RAW_DATA_TYPE_VIDEO);
}
.
.
.
void ZoomSDKRenderer::onRawDataFrameReceived(YUVRawDataI420* data)
{
}
void ZoomSDKRenderer::onRawDataStatusChanged(RawDataStatus status)
{
}
void ZoomSDKRenderer::onRendererBeDestroyed()
{
}

Each frame of video data is available through the YUVRawDataI420 object. Access various pieces of data through this object in onRawDataFrameReceived.

const int width = data->GetStreamWidth();
const int height = data->GetStreamHeight();
const int bufLen = data->GetBufferLen();
const int rotation = data->GetRotation();
const int sourceID = data->GetSourceID();

The Meeting SDK for Linux supports receiving videos in the resolutions listed on the ZoomSDKResolution page in the Meeting SDK for Linux reference docs.

Receive raw audio data

Access mixed and per-user raw audio data through your implementation of IZoomSDKAudioRawDataDelegate. Unlike raw video data, raw audio data defaults to stack-based memory if you don't specify a memory mode.

Mixed raw audio data is combined audio output from one or more users, as heard in a session.

To access raw audio data:

1. Implement an instance of the IZoomSDKAudioRawDataDelegate.
2. Access IZoomSDKAudioRawDataHelper using GetAudioRawdataHelper();.
3. Subscribe to audio using IZoomSDKAudioRawDataHelper.
4. Listen for these callbacks in your listener.

ZoomSDKAudioRawDataDelegate* audio_source = new ZoomSDKAudioRawDataDelegate();
IZoomSDKAudioRawDataHelper* audioHelper;
audioHelper = GetAudioRawdataHelper();
if (audioHelper) {
	SDKError err = audioHelper->subscribe(audio_source);
	if (err != SDKERR_SUCCESS) {
		std::cout << "Error occurred subscribing to audio : " << err << std::endl;
	}
}
else {
	std::cout << "Error getting audioHelper" << std::endl;
}
.
.
.
void ZoomSDKAudioRawData::onOneWayAudioRawDataReceived(AudioRawData* data, uint32_t node_id)
{
}
void ZoomSDKAudioRawData::onMixedAudioRawDataReceived(AudioRawData* data)
{
}

From within the callbacks, access the data buffer with data->GetBuffer().