Creating a Dynamic Layout for Multi Host Broadcasts with Amazon IVS

I’ve written just a few posts these days about multi host reside streams with Amazon Interactive Video Service (Amazon IVS). It is an thrilling characteristic that opens up worlds of potentialities that merely weren’t out there till lately. We first checked out easy methods to create a multi host reside chat utility. Subsequent, we noticed easy methods to broadcast that reside chat session to an Amazon IVS channel.

Once we checked out including chat individuals to the printed consumer in that final publish, you most likely seen that I cheated a bit and hardcoded the VideoComposition values that inform the printed consumer the dimensions and place of the participant’s video on the consumer. Properly – cheated is a robust phrase – to illustrate that I deliberately simplified the code to give attention to the method of broadcasting a reside chat session. Primarily what we’re on the lookout for right here is modifying the dimensions and place of the participant’s video within the broadcast in order that when there’s one video, the structure will look one thing like this:

However when there are two movies, the structure will change to one thing like this:

And when there are 5:

You get the concept – a dynamic structure that modifications based mostly on the quantity of individuals.

On this publish, we’ll check out one method you could possibly make the most of to make making a dynamic structure a bit simpler. We’ll construct off of the answer within the final publish, so if you have not learn that publish but, it is most likely a good suggestion to try this now.

Within the final publish, we listened for an occasion referred to as STAGE_PARTICIPANT_STREAMS_ADDED. Within the occasion handler for that occasion, we added our individuals to the DOM, and rendered the audio and video to the IVSBroadcastClient occasion. With a purpose to render a dynamic structure, we’ll want to trace what number of individuals are at present within the session, so we’ll add an array referred to as participantIds as a worldwide variable. Let’s modify the occasion handler to push the present participant id to that array.

stage.on(StageEvents.STAGE_PARTICIPANT_STREAMS_ADDED, (participant, streams) => {

  //add participant id to array
  participantIds.push(participant.id);

  renderParticipant(participant, streams);
  renderVideosToClient(participant, streams.discover(s => s.streamType === StreamType.VIDEO));
  renderAudioToClient(participant, streams.discover(s => s.streamType === StreamType.AUDIO));
  updateVideoCompositions();
});

Within the final publish, I discussed that the updateVideoCompositions() technique was not proven as a result of the implementation would differ. We’ll speak about one doable implementation in only a bit. For now let’s check out how we are able to get a dynamic structure config as a substitute of hardcoding it as we did within the final publish.

One approach to receive a dynamic measurement and place is to loop over the participant array and calculate them based mostly on the variety of individuals, the dimensions of the <canvas>, and the specified quantity of rows, columns, and padding. However, why? That appears like a variety of troublesome code and pointless work once you notice that these values by no means change. In case you have one participant, the video will probably be a set measurement and centered within the <canvas>. It would not matter what number of individuals get added – the structure for every video will all the time be the identical for a given variety of individuals. So why waste time and CPU cycles once we might pre-calculate these values and retailer them in an array of arrays.

For my demo, I spent a while figuring out the perfect values with an intensive half-hour with a pen, paper and calculator to find out the composition values for every doable structure. Please observe: I used to be not a maths or artwork main as evidenced by the next sketch.

For this demo, I capped my reside stream at solely displaying movies for the primary 6 individuals. Your use case might dictate one thing totally different, however having greater than 6 participant movies in a single reside stream turns into a bit too busy in my expertise.

Right here is the results of my calculations:

const layouts = [
  [{ height: 720, width: 1280, x: 320, y: 180 }],
  [{ height: 450, width: 800, x: 80, y: 315 }, { height: 450, width: 800, x: 1040, y: 315 }],
  [{ height: 450, width: 800, x: 80, y: 45 }, { height: 450, width: 800, x: 1040, y: 45 }, { height: 450, width: 800, x: 560, y: 585 }],
  [{ height: 450, width: 800, x: 80, y: 45 }, { height: 450, width: 800, x: 1040, y: 45 }, { height: 450, width: 800, x: 80, y: 585 }, { height: 450, width: 800, x: 1040, y: 585 }],
  [{ height: 337, width: 600, x: 20, y: 100 }, { height: 337, width: 600, x: 650, y: 100 }, { height: 337, width: 600, x: 1280, y: 100 }, { height: 337, width: 600, x: 340, y: 640 }, { height: 337, width: 600, x: 980, y: 640 }],
  [{ height: 337, width: 600, x: 20, y: 100 }, { height: 337, width: 600, x: 650, y: 100 }, { height: 337, width: 600, x: 1280, y: 100 }, { height: 337, width: 600, x: 20, y: 640 }, { height: 337, width: 600, x: 650, y: 640 }, { height: 337, width: 600, x: 1280, y: 640 }]
];

That may look overwhelming, however take into account that every ingredient within the outer array ingredient comprises an array of compositions for every video. If there are 3 individuals, we are able to reference the third ingredient within the outer array, and the place of the participant id within the participantIds array will decide which composition will apply to that video. We will modify our renderVideosToClient() perform to seize the correct composition and use these values once we add the video to the printed consumer.

const renderVideosToClient = async (participant, stream) => {
  const participantId = participant.id;
  const videoId = `video-${participantId}`;

  // get the index of this participantId
  const pIdx = participantIds.indexOf(participantId);

  let composition = layouts[participantIds.length - 1][pIdx];
  config.index = 2;

  const mediaStream = new MediaStream();
  mediaStream.addTrack(stream.mediaStreamTrack);
  broadcastClient.addVideoInputDevice(mediaStream, videoId, composition);
};

However bear in mind – if we solely do that when a participant is added, the earlier video compositions will nonetheless replicate the composition that was utilized once they had been added. That’s the place the updateVideoCompositions() perform comes into the image. Right here we loop over the participantIds array, seize the correct composition from layouts, and use the updateVideoDeviceComposition() (docs) technique of the broadcastClient.

const updateVideoCompositions = async () => {
  let idx = 0;
  for (const p of participantIds) {
    const videoId = `video-${p}`;
    let config = layouts[filteredParticipantIds.length - 1][idx];
    config.index = 2;
    broadcastClient.updateVideoDeviceComposition(videoId, config);
    idx = idx + 1;
  }
};

We must also ensure that when a participant leaves the stage that we take away the participant id from the array and once more replace the composition for all movies.

stage.on(StageEvents.STAGE_PARTICIPANT_STREAMS_REMOVED, (participant, streams) => {
  const participantId = participant.id;

  // take away participant id from array
  const pIdx = participantIds.findIndex(id => id === participantId);
  participantIds.splice(pIdx, 1);

  const videoTrackId = `video-${participantId}`;
  const audioTrackId = `audio-${participantId}`;
  if (broadcastClient.getVideoInputDevice(videoTrackId)) broadcastClient.removeVideoInputDevice(videoTrackId);
  if (broadcastClient.getAudioInputDevice(audioTrackId)) broadcastClient.removeAudioInputDevice(audioTrackId);
  const videoId = `${participantId}-video`;
  doc.getElementById(videoId).closest('.participant-col').take away();
  updateVideoCompositions(); 
}); 

As talked about above, you will more than likely need to restrict the quantity of movies which are added to the reside stream through the printed consumer. You would possibly need to add a static picture as a substitute of the ultimate video to indicate that there are extra individuals than what are proven:

Abstract

On this publish, we discovered one method for dynamic layouts when broadcasting a multi host stage with Amazon IVS. In a future publish, we’ll have a look at further choices for broadcasting with a number of hosts. As all the time, in case you have any questions or feedback, please go away them beneath.