Video

You can programmatically control user video for an ongoing session using the methods provided by the Video SDK.

Video controls

Using the SDK functions, users in a session can be provided with controls for managing their video. Session hosts can manage their own video as well the video of other users in the session.

Some common features that you might want to include in your app could be an option to enable or disable a user's video. Prior to changing the video control, you may also need to obtain data about the state of each users' present it in the UI of your app.

The following are some examples that you can refer to when implementing features related to video controls.

Check the video status of a user

Video can be controlled on a per-user basis. First, you should check the current video status of the user:

// Get videoStatus for User.
ZoomVideoSDKVideoStatus videoStatus = pUserInfo->getVideoStatus();
// Check if User's video is on.
bool isVideoOn = pUSerInfo->getVideoPipe()->getVideoStatus().isOn;

Start video

If the user's video is not already enabled, you can start video using startVideo, provided by the ZoomVideoSDKVideoHelper:

// Get the IZoomVideoSDKVideoHelper to perform video actions.
IZoomVideoSDKVideoHelper* pVideoHelper = m_pVideoSDK->getVideoHelper();
if (pVideoHelper) {
    // Start local User's video.
    pVideoHelper->startVideo();
}

Stop video

To stop displaying the video of the local user, use stopVideo:

// Get the IZoomVideoSDKVideoHelper to perform video actions.
IZoomVideoSDKVideoHelper* pVideoHelper = m_pVideoSDK->getVideoHelper();
if (pVideoHelper) {
    // Stop local User's video.
    pVideoHelper->stopVideo();
}

Render a user's video

The Video SDK allows you to render the video of each user who join a session.

  1. Retrieve the IZoomVideoSDKUser object related to each user as they join the session.
  2. Obtain the IZoomVideoSDKRawDataPipe from each user whose stream you'd like to render.
  3. Subscribe to their video pipe.
  4. Listen for frames.

Retrieve user

To be notified when users join a session, you can use onUserJoin within ZoomVideoSDKDelegate:

void CExampleListener::onUserJoin(IZoomVideoSDKUserHelper* pUserHelper, IVideoSDKVector<IZoomVideoSDKUser*>* userList)
{
    CString strInfo;
    IZoomVideoSDKUser* pUser;
    int count = userList->GetCount();
    for (int i = 0; i < count; i++)
    {
        pUser = userList->GetItem(i);
        if (!pUser) continue;
        strInfo.Format(_T("A user joined the session: name=%s"), pUser->getUserName());
    }
}

For more information on implementing this delegate, see Integrate.

Subscribe to user's video pipe

Now that you have access to each user represented by a IZoomVideoSDKUser object, you must retrieve the IZoomVideoSDKRawDataPipe of each user whose video you would like to render and to subscribe to that user's video pipe.

// CExampleRenderer.h
class CExampleRenderer : public IZoomVideoSDKRawDataPipeDelegate
{
    // ...
public: // IZoomVideoSDKRawDataPipeDelegate
    virtual void onRawDataFrameReceived(YUVRawDataI420* data_);
    virtual void onRawDataStatusChanged(RawDataStatus status);
    // ...
}
// CExampleRenderer.cpp
ZoomVideoSDKErrors CExampleRenderer::Subscribe(IZoomVideoSDKUser* pUser, ZoomVideoSDKRawDataType dataType, int size)
{
    // Set the resolution.
    ZoomVideoSDKResolution resolution = ZoomVideoSDKResolution_360P;
    // Get the video pipe for the user.
    IZoomVideoSDKRawDataPipe* pPipe = NULL;
    pPipe = pUser->GetVideoPipe();
    if (!pPipe) return;
    // Call subscribe.
    err = pPipe->subscribe(resolution, this)
    return err;
}

Receive video frames

The class above inherits from IZoomVideoSDKRawDataPipeDelegate, which provides two callbacks. onRawDataFrameReceived(YUVRawDataI420*data) provides video data in YUV format. This data parameter YUVRawDataI420 provides everything needed to be able to render the single video frame's data.

void CExampleRenderer::onRawDataFrameReceived(YUVRawDataI420* data_)
{
    // Get frame data resolution.
  data_->GetStreamWidth();
    data_->GetStreamHeight();
    // Get frame buffer.
    data_->GetYBuffer();
    data_->GetUBuffer();
    data_->GetVBuffer();
    // Get frame rotation
    data_->GetRotation();
}

The SDK calls onRawDataStatusChanged(RawDataStatus status) when there is a change in raw data status.

void CExampleRenderer::onRawDataStatusChanged(RawDataStatus status)
{
    if (status == RawData_On)
    {
        // Now subscribed to user's data.
    }
    else
    {
        // No longer subscribed to user's data.
    }
}

Unsubscribe from User's Video Pipe

To stop rendering a user's content, you must call unsubscribe() to unsubscribe from the user's video.

void CExampleRenderer::unSubscribe(IZoomVideoSDKUser* pUser)
{
    if (pUser->GetVideoPipe())
        pUser->GetVideoPipe()->unSubscribe(this);
}

Use virtual background

Use IZoomVideoSDKVideoHelper to add, get, set, and remove virtual backgrounds.

To add virtual backgrounds:

  1. Use addVirtualBackgroundItem to add a virtual background item.
  2. Use getVirtualBackgroundItemList to get a list of available virtual background items.
  3. Choose a virtual background item from the list and use setVirtualBackgroundItem to set the virtual background.

Use removeVirtualBackgroundItem to remove the virtual background or set the virtual background to None. Use getSelectedVirtualBackgroundItem to get the current selected background.

virtual ZoomVideoSDKErrors addVirtualBackgroundItem(const zchar_t *image_path, IVirtualBackgroundItem **imageItem) = 0
    // Add virtual background object.
virtual ZoomVideoSDKErrors removeVirtualBackgroundItem(IVirtualBackgroundItem * imageItem) = 0
    // Remove virtual background object.
virtual IVideoSDKVector<IVirtualBackgroundItem * > * getVirtualBackgroundItemList() = 0
    // Get virtual background item list. Call this function first before creating a virtual background object.
virtual ZoomVideoSDKErrors setVirtualBackgroundItem(IVirtualBackgroundItem * imageItem) = 0
    // Set virtual background item.
virtual IVirtualBackgroundItem * getSelectedVirtualBackgroundItem() = 0
    // Get virtual background item which has been selected.

Sending raw video data (optional)

The class above inherits from IZoomVideoSDKRawDataPipeDelegate, which provides two callbacks. onRawDataFrameReceived(YUVRawDataI420*data provides video data in YUV format. This data parameter YUVRawDataI420 provides everything needed to render the single video frame's data.

// CExampleVideoSource.h
class CExampleVideoSource : public IZoomVideoSDKVideoSource
{
    // ...
public: // IZoomVideoSDKVideoSource
    virtual    void onInitialize(IZoomVideoSDKVideoSender* sender, IVideoSDKVector<VideoSourceCapability >* support_cap_list, VideoSourceCapability& suggest_cap);
    virtual void onPropertyChange(IVideoSDKVector<VideoSourceCapability >* support_cap_list, VideoSourceCapability suggest_cap);
    virtual void onStartSend();
    virtual void onStopSend();
    virtual void onUninitialized();
    // ...
private:
    IZoomVideoSDKVideoSender* m_pSender;
}
// CExampleVideoSource.cpp
void CExampleVideoSource::onInitialize(IZoomVideoSDKVideoSender* sender, IVideoSDKVector<VideoSourceCapability >* support_cap_list, VideoSourceCapability& suggest_cap)
{
    // Store IZoomVideoSDKVideoSender.
    if (!sender) return;
    m_pSender = sender;
    // Inspect video capabilities.
    if (!support_cap_list) return;
    unsigned int nCount = support_cap_list->GetCount();
    for(int i=0; i<nCount; i++)
    {
        VideoSourceCapability& cap_ = support_cap_list->GetItem(i);
    }
}
// Call sendVideoFrame to send a frame buffer of raw data.
char* pFrameData;
int width;
int height;
int frameLength;
int rotation;
m_pSender->sendVideoFrame(pFrameData, width, height, frameLength, rotation);

The support_cap_list parameter provides a vector of current video capabilities for each frame to send. The suggest_cap parameter is the current suggested video capability. The callback onPropertyChange will be called when this changes.

void CExampleVideoSource::onPropertyChange(IVideoSDKVector<VideoSourceCapability >* support_cap_list, VideoSourceCapability suggest_cap)
{
    if (!support_cap_list) return;
    unsigned int nCount = support_cap_list->GetCount();
    for(int i=0; i<nCount; i++)
    {
        VideoSourceCapability& cap_ = support_cap_list->GetItem(i);
    }
    // Example: Suggested video resolution changed to 1080p.
    if (suggest_cap.width == 1920 && suggest_cap.height == 1080)
    {
    }
}

You can also preprocess raw video data using onPreProcessRawData within IZoomVideoSDKVideoSourcePreProcessor.

void CExampleVideoSourcePreProcessor::onPreProcessRawData(YUVProcessDataI420* rawData)
{
    // Use the rawData parameter to perform preprocessing actions.
    unsigned int frameWidth = rawData->GetWidth();
    unsigned int frameHeight = rawData->GetHeight();
    char* yBuffer = rawData->GetYBuffer();
    char* uBuffer = rawData->GetUBuffer();
    char* vBuffer = rawData->GetVBuffer();
}

Camera Controls

Configure camera functionality using IZoomVideoSDKVideoHelper. To get the current camera list use getCameraList.

IZoomVideoSDKVideoHelper* pInsVideoHelper = m_pVideoSDK->getVideoHelper();
if (pInsVideoHelper->getNumberOfCameras() > 0)
    {
        IVideoSDKVector<IZoomVideoSDKCameraDevice*>* pCameraList = pInsVideoHelper->getCameraList();
        for (int i=0; i<pCameraList->GetCount(); i++)
        {
            IZoomVideoSDKCameraDevice* pCamera = m_pCameraList->GetItem(i);
            CString strItem;
            strItem.Format(_T("%s***%s"), pCamera->getDeviceName(), pCamera->getDeviceId());
    }
    }

To select a camera use selectCamera on the IZoomVideoSDKCameraDevice object of the desired camera.

IVideoSDKVector<IZoomVideoSDKCameraDevice*>* pCameraList = pInsVideoHelper->getCameraList();
IZoomVideoSDKCameraDevice* pCamera = m_pCameraList->GetItem(0);
pVideoHelper->selectCamera(pCamera->getDeviceId());

You can also switch to the next available camera with switchCamera.

IZoomVideoSDKVideoHelper* pVideoHelper = m_pVideoSDK->getVideoHelper();
if (!pVideoHelper) return;
pVideoHelper->switchCamera();

Video callbacks

Be sure that you have set up a delegate for callback events to receive video callbacks.

Get notified when a user's video status has changed

void CExampleListener::onUserVideoStatusChanged(IZoomVideoSDKVideoHelper* pVideoHelper, IVideoSDKVector<IZoomVideoSDKUser*>* userList)
{
    CString strInfo;
    IZoomVideoSDKUser* pUser;
    int count = userList->GetCount();
    for (int i = 0; i < count; i++)
    {
        pUser = userList->GetItem(i);
        strInfo.Format(_T("A user's video status changed: userid=%s, name=%s, status=%s"), pUser->getUserName(), pUser->getVideoStatus());
    }
}

Video quality preference

When network bandwidth is limited, you can adjust video quality preferences between resolution and frame rate. For example, you can choose to preserve the video sharpness or smoothness. If bandwidth is not a concern, you can receive the best quality video using a high video resolution and maximum frame rate.

Set your video quality preferences with the tagVideoPreferenceSetting.

Video preference modes

Choose from the following modes, depending on what you'd like to prioritize:

  • Balance mode. Zoom will do what is best under the current bandwidth situation and make adjustments as needed. You don't need to set any additional parameters for this mode. This mode is suitable for video conference usage. This is the default preference.
  • Smoothness mode. Preserves the frame rate as much as possible. If network bandwidth degrades, Zoom will sacrifice video resolution to preserve the frame rate. This prioritizes a smooth video frame transition.
  • Sharpness mode. Preserves the resolution as much as possible. If network bandwidth degrades, Zoom will sacrifice the frame rate to preserve video resolution. This prioritizes a sharp video image.
  • Custom mode. Allows you to provide the minimum and maximum frame rate. Use this mode if you have an understanding of your network behavior and a clear idea of how to adjust the frame rate to achieve the desired video quality. You can also use this mode to influence bandwidth usage by increasing or decreasing the maximum frame rate setting. Given a resolution, a lower maximum frame rate results in less bandwidth usage. Note that if the bandwidth cannot be sustained by following the minimum and maximum frame rates, the system will drop down to the next lower resolution.

See the reference documentation for details.

Use cases

Video quality preferences are useful when the primary focus is not the image of a person attending the session, but on other video images where smoothness or sharpness is preferred.

Aside from video conferencing, you may want to set video quality preferences for:

  • Transmission of medical X-ray images.
  • Transmission of video captured by endoscopic cameras during a medical operation.
  • Live sporting event broadcasts, such as co-watching or broadcasting a football game.