How to stream audio from Amazon Chime SDK meeting to Amazon Kinesis Video Streams?

0

I am building a web application that hosts video meetings using the Amazon Chime SDK. I want to stream the audio from each meeting to Amazon Kinesis Video Streams (KVS) for real-time processing.

Amazon Chime media capture pipelines are not an option for our use case due to their limitation of only supporting 10 concurrent pipelines. (https://docs.aws.amazon.com/general/latest/gr/chime-sdk.html#media-pipeline-quotas)

I understand that the Chime SDK has an audio analysis feature that can be used to stream audio to KVS, but I’m looking for clear implementation guidance. (This approach was already suggested to me: https://repost.aws/questions/QUSakm93SnS76RbZa4g9KC3A/how-to-stream-audio-from-amazon-chime-sdk-video-meetings-to-kinesis-video-streams-for-real-time-transcription )

I found information that audio session can be sent to KVS but I could not find any implementation help:

"Send your WebRTC session audio to Amazon Kinesis Video Streams to perform live analysis AWS ML services. Send each attendees’ audio stream to a separate Kinesis Video Stream for individual analysis, or send a single mixed-audio stream with the meetings audio. " https://aws.amazon.com/chime/chime-sdk/features/

I would like to know: How can I stream audio (single mixed-audio stream) from a Chime SDK meeting to Kinesis Video Streams?

Thank you!

2 Answers
0

Please note that the default limits are per-region and us-east-1 allows 100 (not 10) concurrent media pipelines by default. The other regions are 10 each. https://docs.aws.amazon.com/chime-sdk/latest/dg/media-pipelines-limits.html

Also please note that these are just the default SOFT limits. These quotas can be adjusted to accommodate your business requirements. Please make a request through the Service Quotas page in the AWS Console.

For sending meeting audio to KVS, media capture pipelines are the appropriate solution and you should not let the default limit prevent you from proceeding with your application.

However, if you are only processing the audio for transcription, you will get better results using the native integration with Amazon Transcribe. The Amazon Chime SDK live transcription feature will manage this for you. (https://docs.aws.amazon.com/chime-sdk/latest/dg/meeting-transcription.html)

profile pictureAWS
answered 12 days ago
  • Thanks for your response. Do I understand correctly that there are not any other solution than media capture pipelines to stream audio to KVS?

    Regarding the soft limit: Before finalizing our solution architecture, I need to ensure this limit won’t become a bottleneck as we scale. Specifically, is it feasible to raise the limit to support thousands of concurrent meetings in the future?

  • The media capture pipeline is the only way provided directly by the Amazon Chime SDK and API. With some effort on your part, though, you could implement an SDK client app (which would receive the meeting audio) that is also a KVS producer (which would send the meeting audio to KVS). You'd have to build and manage this to scale. The media capture pipelines do all that on your behalf.

    Each limit increase is evaluated individually. While I can't be certain what would be done in your case, there is not a hard upper limit. With the proper justification and a trend of growing usage, very high limits of various quotas have been provided to customers.

-1

To stream audio from an Amazon Chime SDK meeting to Amazon Kinesis Video Streams (KVS), you can use the media stream pipeline feature. Here's a high-level implementation guide:

  1. Create a Kinesis Video Streams pool: First, you need to create a KVS pool by calling the CreateMediaPipelineKinesisVideoStreamPool API. This pool must be in the same AWS account as your Chime SDK meeting.

  2. Set up the media stream pipeline: Once you have created the KVS pool, you can create a media stream pipeline for your meeting. You can create one media stream pipeline per Amazon Chime SDK meeting.

  3. Configure the pipeline: When setting up the pipeline, you can configure it to capture a single mixed-audio stream with the meeting's audio, which aligns with your requirement.

  4. Start the pipeline: After configuration, start the pipeline to begin streaming the audio to KVS.

  5. Access the streamed data: The audio data will be stored in MKV format with AAC encoding in the Kinesis Video Stream. You can access this data using KVS APIs like GetMedia or GetMediaForFragmentList.

  6. Process the audio: Once the audio is in KVS, you can perform real-time processing, such as transcription or other analysis.

Keep in mind that the KVS pool operation (creating, updating, and deleting streams) is asynchronous. You'll receive EventBridge notifications about the status of streams in the pool.

This approach allows you to bypass the limitations of media capture pipelines and support more than 10 concurrent meetings. The media stream pipelines automatically scale based on your concurrent call volume, making it suitable for applications requiring support for multiple simultaneous meetings.

Remember to manage your KVS pools efficiently. You can reuse a pool across different meetings and delete pools when they're no longer needed. Always ensure you've ended all meetings using a pool before attempting to delete it.
Sources
Creating media stream pipelines using the Amazon Chime SDK - Amazon Chime SDK
Creating a Kinesis Video Streams pool for Amazon Chime SDK media stream pipelines - Amazon Chime SDK
Using Amazon Chime SDK media stream pipelines data - Amazon Chime SDK

profile picture
answered 13 days ago
  • This is not a feasible solution because it suggests to use Amazon Chime media capture pipelines but those are not an option for us because those have limitation of 10 concurrent meetings.

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions