Receive raw data

You receive a remote user's raw video and share frames by subscribing to their raw data pipe, and raw audio through the audio helper.

Receive raw video data

Raw video data is encoded in the YUV420p format, commonly used by an OpenGL ES based renderer. To access and modify the video data, follow these steps.

  1. Implement an instance of IZoomVideoSDKRawDataPipeDelegate.
  2. Use the callbacks it provides to receive each frame of raw video data.
  3. Pass the delegate to the video pipe of a specific user with GetVideoPipe.
class CExampleRenderer : public IZoomVideoSDKRawDataPipeDelegate
{
public:
    virtual void onRawDataFrameReceived(YUVRawDataI420* data_);
    virtual void onRawDataStatusChanged(RawDataStatus status);
};
IZoomVideoSDKRawDataPipe* pPipe = pUser->GetVideoPipe();
if (pPipe) {
    pPipe->subscribe(ZoomVideoSDKResolution_360P, pRenderer);
}

Each frame is delivered through the YUVRawDataI420 object in onRawDataFrameReceived.

void CExampleRenderer::onRawDataFrameReceived(YUVRawDataI420* data_)
{
    const int width = data_->GetStreamWidth();
    const int height = data_->GetStreamHeight();
    const int bufLen = data_->GetBufferLen();
    const int rotation = data_->GetRotation();
    const int sourceID = data_->GetSourceID();
    // Access the frame buffers.
    char* yBuffer = data_->GetYBuffer();
    char* uBuffer = data_->GetUBuffer();
    char* vBuffer = data_->GetVBuffer();
}

The SDK calls onRawDataStatusChanged when the pipe's data status changes.

void CExampleRenderer::onRawDataStatusChanged(RawDataStatus status)
{
    if (status == RawData_On) {
        // Now subscribed to the user's data.
    } else {
        // No longer subscribed to the user's data.
    }
}

To stop receiving frames, call unSubscribe with the same delegate.

pUser->GetVideoPipe()->unSubscribe(pRenderer);

The Video SDK for Windows supports receiving video at the resolutions enumerated for ZoomVideoSDKResolution in the reference.

Receive raw audio data

Through your implementation of IZoomVideoSDKDelegate, you can access mixed audio (the combined output of all users, as heard in a session) and per-user raw audio.

Unlike raw video data, raw audio data will default to stack-based memory if you do not specify a memory mode.

  • The virtual speaker allows access to audio data received from other users in the session. This data represents what a user would hear played through the device's speakers.
  • The virtual mic allows audio data for the current user to be sent to the session programmatically instead of from the SDK capturing it through an audio input device.

To access raw audio data:

  1. Get the IZoomVideoSDKAudioHelper with getAudioHelper.
  2. Connect to audio with startAudio.
  3. Subscribe to audio with subscribe.
  4. Listen for the callbacks in your delegate.
IZoomVideoSDKAudioHelper* pAudioHelper = m_pVideoSDK->getAudioHelper();
pAudioHelper->startAudio();
pAudioHelper->subscribe();
void CExampleListener::onMixedAudioRawDataReceived(AudioRawData* data_)
{
    // Access the mixed audio for the whole session.
}
void CExampleListener::onOneWayAudioRawDataReceived(AudioRawData* data_, IZoomVideoSDKUser* pUser)
{
    // Access the audio for a single user.
}

From within the callbacks, access the data buffer with data_->GetBuffer(). For the full list of audio callbacks, see Listen for audio events.

Receive raw audio for the virtual speaker

Use the virtual speaker to process audio sent to the speaker. To receive raw audio that was sent through IZoomVideoSDKVirtualAudioMic:

  1. Implement an instance of IZoomVideoSDKVirtualAudioSpeaker.
  2. Pass that implementation into ZoomVideoSDKSessionContext before joining a session.
  3. Access raw data in each callback method.
ZoomVideoSDKVirtualAudioSpeaker* pSpeaker = new ZoomVideoSDKVirtualAudioSpeaker();
session_context.virtualAudioSpeaker = pSpeaker;
void ZoomVideoSDKVirtualAudioSpeaker::onVirtualSpeakerMixedAudioReceived(AudioRawData* data_)
{
    // Process the mixed audio sent to the speaker.
}

Receive raw share data

You can receive raw share video and audio data, for example when someone is sharing their screen.

Receive raw share video data

Receiving share video follows the same pattern as receiving raw video, except you get the pipe from the user's share action rather than from GetVideoPipe. Get the user's share actions with getShareActionList, then subscribe to the pipe from the share action you want.

IVideoSDKVector<IZoomVideoSDKShareAction*>* pShareActions = pUser->getShareActionList();
if (pShareActions && pShareActions->GetCount() > 0)
{
    IZoomVideoSDKShareAction* pShareAction = pShareActions->GetItem(0);
    IZoomVideoSDKRawDataPipe* pSharePipe = pShareAction->getSharePipe();
    if (pSharePipe) {
        pSharePipe->subscribe(ZoomVideoSDKResolution_360P, pRenderer);
    }
}

Receive raw share audio data

Receive raw share audio through the onSharedAudioRawDataReceived callback, the same way you receive raw audio. For details, see Listen for audio events.