Manipulating the UI

The following information is applicable to Zoom Apps Layers API v1.5, and requires Zoom Client v5.10.6 or higher.

You can use the Zoom Apps layers API to control the UI for your app.

Drawing the app onto the canvas

A Zoom Apps app can directly control the background and can generate a wide range of outputs in the WebView. This can be a simple static background image or complex, dynamically-generated output with interactivity. When you call runRenderingContext() for an immersive mode the app is drawn onto the meeting canvas. When you call runRenderingContext() for Camera Mode the app is drawn into an off-screen rendering engine and only shown in your participants' video stream.

Drawing participants in Immersive Modes

You can use the drawParticipant method to draw participant videos and static images within the canvas.

Usage example

zoomSdk.drawParticipant({
  participantUUID: 'xxx',
  x: 0,
  y: 0,
  width: 1280,
  height: 720,
  zIndex:1,
  cutout: 'rectangle' || `person'
})
.then((ctx) => {
    console.log("drawParticipant returned", ctx);
})
.catch((e) => {
    console.log(e);
});

Drawing participants in Camera Mode

You can use the drawParticipant method to draw participant videos and static images within the canvas. In camera mode, the drawParticipant has these conditions:

  • The current participant's video may be placed within the rendering region (unlike immersive mode, other participants may NOT be drawn).
  • All x/y/width/height values are raw pixels, relative to the renderTarget dimensions returned by config().
  • The config() API's response includes media.renderTarget.width and media.renderTarget.height properties, to specify the size of the virtual camera video frame [default 1280x720].

Usage example

zoomSdk
    .drawParticipant({
        participantUUID: "xxx",
        x: 0,
        y: 0,
        width: 1280,
        height: 720,
        zIndex: 1,
    })
    .then((ctx) => {
        console.log("drawParticipant returned", ctx);
    })
    .catch((e) => {
        console.log(e);
    });

Drawing images in Immersive Modes

You can use the drawImage method to draw static images onto the canvas at any x,y,z index.

Usage example

zoomSdk.drawImage({
  imageData: <ImageData (includes width/height properties)>,
  x: 0, y: 0, zIndex:3
})
.then((ctx) => {
    console.log("drawImage returned", ctx);
})
.catch((e) => {
    console.log(e);
});

Constraints

Keep the following in mind when using drawImage:

  • The drawImage() method does not directly support HiDPi image sizes.

    You cannot draw an image the size of the screen so you need to tile the screen.

  • If you want to support HiDPi screens you need to factor in the scaling ratio:

    • Draw to the on-screen or in-memory canvas using the scaling ratio.
    • Divide out the scaling ratio for coordinates when drawing Image Data to Zoom.
    • Keep the scaling ratio for the width and height.

Drawing images in Camera Mode

You can use the drawImage method in camera mode.

Usage example

zoomSdk.drawImage({
  imageData: <ImageData (includes width/height properties)>,
  x: 0, y: 0, zIndex:3
})
.then((ctx) => {
    console.log("drawImage returned imageID", ctx.imageID);
})
.catch((e) => {
    console.log(e);
});

Adjusting and clearing images

To move, resize, or otherwise adjust an already drawn object, the app must use clearImage or clearParticipant to clear the existing object and then set it again using drawImage or drawParticpant as appropriate.

Usage example for clearing participants

zoomSdk
    .clearParticipant({
        participantUUID: "xxx",
    })
    .catch((e) => {
        console.log(e);
    });

Usage example for clearing images

zoomSdk
    .clearImage({
        imageId: "xxx",
    })
    .catch((e) => {
        console.log(e);
    });

Additional methods for Camera Mode

Drawing webviews

You can specify the size, placement and z-order of the OSR webview (there is only one webview):

zoomSdk
    .drawWebView({
        webviewId: "xxx",
        x: 0,
        y: 0,
        width: 1280,
        height: 720,
        zIndex: 2,
    })
    .then((ctx) => {
        console.log("drawWebView returned", ctx);
    })
    .catch((e) => {
        console.log(e);
    });

Clearing webviews

The WebView may be hidden (the Camera Mode app remains running):

zoomSdk
    .clearWebView({
        webviewId: "xxx",
    })
    .catch((e) => {
        console.log(e);
    });

Sending messages (app-to-app)

Using postMessage for the two app instances (side panel app : camera mode app).

zoomSdk
    .postMessage({})
    .then(function (response) {})
    .catch(function (error) {
        console.log(e);
    });

Using Controller Mode with all Modes

The postMessage method posts messages between multiple app instances running in a meeting. You can use it for messaging between already-running instances of the same app.

  • For this use case, connect() is not required.
  • The API definition is: postMessage([json object]).

Usage example

zoomSdk.postMessage({ command: "hello", testFlag: true });

onMessage event

The onMessage event supports messages from other instances of the same app.

Usage example

zoomSdk.addEventListener("onMessage", (eventInfo) => {
    console.log("received:", eventInfo);
});

Managing changes in the UI

  • Resizing windows

    When the Zoom Client's meeting window is resized, the app is responsible for moving and resizing participants and images.

    Note: This is not relevant to Camera Mode.

  • Events

    The onMyMediaChange event fires when the app user's video changes. For example, when the user chooses a different camera, toggles "Original ratio", or toggles "HD" in video settings. The size is in device-based pixels of the source video (not CSS pixels).

    To provide information about the original size of the app's video, the zoomSdk.config response contains a media object alongside the runningContext and clientVersion specifying the original video size.

    {
  // ...
  "media": {
    "video": {
      "width": 1920,
      "height": 1080
    }
  }
  // ...
}

Communicating between apps

Apps may use app-specific mechanisms, such as WebSockets to backend services, to inform other apps about layers-related changes. Apps may choose to present the same or different views to the meeting participants, as appropriate for the app.