# Raw data

<!-- WARNING: THIS FILE WAS AUTOGENERATED! DO NOT EDIT! -->

The Video SDK provides you with an option to access real-time raw audio, video, and share data during a session, so you can apply additional effects and enhance the user experience. Use a processor component to apply effects or transformations to each piece of data in real-time.

An example of raw share data is when a user shares their screen.

-   Receive and send raw share video data using the same logic as video data, except get the share pipe instead of the video pipe in the raw data pipe method.
-   Receive and send raw share audio data using the same method used for raw audio data.

Follow the process outlined in this flowchart to capture and process raw data.

![Flowchart outlining the raw data flow](/img/raw-data-flow.png)

## Select raw data memory mode

In order to utilize raw data of any type, you must first select the memory mode you wish to use. The SDK supports heap-based and stack-based memory modes.

### Stack-based memory

-   Variables are allocated automatically and deallocated after the data leaves scope.
-   Variables are not accessible from or transferable to other threads.
-   Typically has faster access than heap-based memory allocation.
-   Memory space is managed by the CPU and will not become fragmented.
-   Variables cannot be resized.

See [Stack-based memory allocation](https://en.wikipedia.org/wiki/Stack-based_memory_allocation) for details.

Specify with:

`ZoomVideoSDKRawDataMemoryModeStack`

### Heap-based memory

-   Variables are allocated and deallocated manually. You must allocate and free variables to avoid memory leaks.
-   Variables can be accessed globally.
-   Has relatively slower access than stack-based memory allocation.
-   Has no guarantee on the efficiency of memory space and can become fragmented.
-   Variables can be resized.

See [Heap-based dynamic memory allocation](https://en.wikipedia.org/wiki/Memory_management#DYNAMIC) for details.

Specify with:

`ZoomVideoSDKRawDataMemoryModeHeap`

## Specify memory mode

After determining which memory mode is right for you, you must specify it when the SDK is initialized. Note that this must be done for audio. video, and share individually. To specify a raw data memory mode, provide one of these enums cases to the `ZoomVideoSDKInitParams` during SDK initialization.

```cpp
ZoomVideoSDKInitParams initParams;
// Set audio memory mode to stack.
initParams.audioRawDataMemoryMode = ZoomVideoSDKRawDataMemoryModeStack;
// Set video memory mode to stack.
initParams.videoRawDataMemoryMode = ZoomVideoSDKRawDataMemoryModeStack;
// Set share memory mode to stack.
initParams.shareRawDataMemoryMode = ZoomVideoSDKRawDataMemoryModeStack;
```

To use heap-based memory, use `ZoomVideoSDKRawDataMemoryModeHeap`.

## Receive raw video data

Raw video and share data is encoded in the YUV420p format. YUV420 is a data object commonly used by the renderer based on OpenGL ES. To access and modify the video data, you will need to do the following:

1. Implement an instance of `IZoomVideoSDKRawDataPipeDelegate`.
2. Use callback functions provided by `IZoomVideoSDKRawDataPipeDelegate` to receive each frame of the raw video data.
3. Pass the delegate into the video pipe of a specific user.

```cpp
ZoomVideoSDKRawDataPipeDelegate *dataDelegate = new ZoomVideoSDKRawDataPipeDelegate();
void ZoomVideoSDKRawDataPipeDelegate::onRawDataFrameReceived(YUVRawDataI420 *data) {
    // Add your code, for example, to store the raw data to a file, send it to be processed, etc.
}

void ZoomVideoSDKRawDataPipeDelegate::onRawDataStatusChanged(RawDataStatus status) {
    // Add your code to process this.
}

ZoomVideoSDKRawDataPipe pipe = user.getVideoPipe();
pipe.subscribe(ZoomVideoSDKVideoResolution.VideoResolution_360P, dataDelegate);
```

Each frame of video data will be made available through the `YUVRawDataI420` object. Various pieces of data can be accessed through this object in `onRawDataFrameReceived`:

```cpp
const int width = data->GetStreamWidth();
const int height = data->GetStreamHeight();
const int bufLen = data->GetBufferLen();
const int rotation = data->GetRotation();
const int sourceID = data->GetSourceID();
```

The Video SDK for Windows supports receiving videos in the resolutions enumerated in `ZoomSDKVideoResolution`. You can find more information in the SDK reference.

## Receive raw audio data

Through your implementation of `IZoomVideoSDKDelegate`, you can access mixed (combined audio output from one or more users, as heard in a session) and per-user raw audio data.

> Unlike raw video data, raw audio data will default to stack-based memory if you do not specify a memory mode.

-   The **virtual speaker** allows access to audio data received from other users in the session. This data represents what a user would hear played through the device's speakers.
-   The **virtual mic** allows audio data for the current user to be sent to the session programmatically instead of from the SDK capturing it through an audio input device.

You can receive raw audio if it was sent through `IZoomVideoSDKVirtualAudioMic`. To do so:

1. Implement an instance of `IZoomVideoSDKVirtualAudioSpeaker`.
2. Pass that implementation into `ZoomVideoSDKSessionContext`.
3. Access raw data in each callback method.

```cpp
ZoomVideoSDKVirtualAudioSpeaker *vSpeaker = new ZoomVideoSDKVirtualAudioSpeaker();
session_context.virtualAudioSpeaker = vSpeaker;
```

To access raw audio data, you must:

1. Access `IZoomVideoSDKAudioHelper` using `getAudioHelper();`.
2. `startAudio` using `IZoomVideoSDKAudioHelper`.
3. Subscribe to audio using `IZoomVideoSDKAudioHelper`.
4. Listen for the following callbacks in your listener:

```cpp
IZoomVideoSDKAudioHelper *m_pAudiohelper = video_sdk_obj->getAudioHelper();
m_pAudiohelper->startAudio();
m_pAudiohelper->subscribe();

virtual void onMixedAudioRawDataReceived(AudioRawData *data_){
    // Access mixed data for the whole session here
};

virtual void onOneWayAudioRawDataReceived(AudioRawData *data_, IZoomVideoSDKUser *pUser){
    // Access user-specific raw data here for the user associated with the userId
};
```

## Receive raw share data

Receive raw share video and audio data, for example, when someone is sharing their screen, similarly to how you receive raw video and raw audio data.

### Receive raw share video data

Follow the same steps and code to [receive raw video data](#receive-raw-video-data), except get the share data instead of the video data, see the following code for an example.

```cpp
// Previous code same as receive raw video

ZoomVideoSDKRawDataPipe pipe = user.GetSharePipe();
pipe.subscribe(ZoomVideoSDKVideoResolution.VideoResolution_360P, dataDelegate);
```

### Receive raw share audio data

Receive raw share audio data in the same way as you [receive raw audio data](#receive-raw-audio-data).

## Send raw video data

In addition to receiving and processing raw video data, you can also send raw or processed video data of a user from the user's device. To do so:

1. Implement an instance of `IZoomVideoSDKVideoSource`.
2. Pass that implementation in `externalShareSource()` in `IZoomVideoSDKSessionContext` before joining a session.
3. Obtain `IZoomVideoSDKVideoSender` from the `onInitialize` callback.
4. Use `sendVideoFrame` in `onStartSend` to send raw video data.

```cpp
ZoomVideoSDKVideoSource *virtual_video_source = new ZoomVideoSDKVideoSource();
session_context.externalVideoSource = virtual_video_source;

void ZoomVideoSDKVideoSource::onInitialize(IZoomVideoSDKVideoSender *sender, IVideoSDKVector<VideoSourceCapability> *support_cap_list, VideoSourceCapability &suggest_cap)
{
    this->video_sender_ = sender;
}

void ZoomVideoSDKVideoSource::onPropertyChange(IVideoSDKVector<VideoSourceCapability> *support_cap_list, VideoSourceCapability suggest_cap) {}

void ZoomVideoSDKVideoSource::onStartSend()
{
    sendVideoFrame(char *frameBuffer, int width, int height, int frameLength, int rotation)
}

void ZoomVideoSDKVideoSource::onStopSend() {}
void ZoomVideoSDKVideoSource::onUninitialized() {}
```

In the `onInitialize(IZoomVideoSDKVideoSender *sender, IVideoSDKVector<VideoSourceCapability> *support_cap_list, VideoSourceCapability &suggest_cap)` callback:

-   The `support_cap_list` parameter is the supported capability list. It combines the supported resolution of the session and the device to create a list of supported capabilities.
-   The `suggest_cap` parameter is the suggested capability. It combines the maximum capability of the session itself and the device to get the real maximum as the suggested capability.

## Send raw audio data

In addition to receiving and processing raw audio data, you can also send raw or processed audio data of a user from the user's device. To do so:

1. Implement an instance of `IZoomVideoSDKVirtualAudioMic`.
2. Pass that implementation into `ZoomVideoSDKSessionContext` before joining a session.
    - Ensure that `audioOption.mute = false; session_context.audioOption.connect = true; in ZoomVideoSDKSessionContext`
3. Obtain `IZoomVideoSDKAudioSender` from the `onMicInitialize` callback.
4. Use `send` in `onMicStartSend` to send raw audio data.

```cpp
ZoomVideoSDKVirtualAudioMic virtualMic = new ZoomVideoSDKVirtualAudioMic()
{
    session_context.virtualAudioMic = virtualMic;
    session_context.audioOption.connect = true;
    session_context.audioOption.mute = false;

    void ZoomVideoSDKVirtualAudioMic::onMicInitialize(ZOOM_VIDEO_SDK_NAMESPACE::IZoomVideoSDKAudioSender * rawdata_sender)
    {
        // Store rawdata_sender for later use
    }

    void ZoomVideoSDKVirtualAudioMic::onMicStartSend()
    {
        // You can now send audio
        rawdata_sender->send(rawData, lengthInBytes, sampleRate);
    }

    void ZoomVideoSDKVirtualAudioMic::onMicStopSend()
    {
        // You can no longer send audio
    }

    void ZoomVideoSDKVirtualAudioMic::onMicUninitialized()
    {
        // Mic is no longer active
    }
}
```

## Send raw share data

You can also send raw screen share data. To do so:

1. Implement an instance of `IZoomVideoSDKShareSource`.
2. Pass that implementation into `startSharingExternalSource()` found in `IZoomVideoSDKShareHelper` when you are in a session.
3. Call `getShareHelper()` to access `IZoomVideoSDKShareHelper`.
4. Use `sendShareFrame` in `onShareSendStarted` to send raw screen share data.

```cpp
ZoomVideoSDKShareSource *virtual_share_source = new ZoomVideoSDKShareSource();
ZoomVideoSDKErrors err2 = video_sdk_obj->getShareHelper()->startSharingExternalSource(virtual_share_source);

void ZoomVideoSDKShareSource::onShareSendStarted(IZoomVideoSDKShareSender *pSender){
    sendShareFrame(char *frameBuffer, int width, int height, int frameLength)};

void ZoomVideoSDKShareSource::onShareSendStopped(){};
```