Low Latency Live streaming with CMAF

1

Hello,

We are trying to achieve low latency live HLS streaming.

Our current setup is pretty common:

  1. On-Premises OBS studio, with encoding tuned for zero latency
  2. Upstreaming via RTMP PUSH to AWS Elemental MediaLive
  3. MediaLive output group to MediaStore. Repeating the same encoding settings (zero latency)
  4. MediaStore
  5. CloudFront CDN with hooked MediaStore origin
  6. HLS.js player tuned for playback buffer minification
    There are great articles from Amazon experts regarding how low/ultra-low latency can be achieved.
    This is one we used to tune our current setup:
    https://aws.amazon.com/blogs/media/part-3-how-to-compete-with-broadcast-latency-using-current-adaptive-bitrate-technologies/

Here is one another great article we found:
https://aws.amazon.com/blogs/media/part-2-getting-started-with-ultra-low-latency-using-aws-elemental-live/
As far as we see that, Common Media Application Format might be helpful to achieve ultra-low latency as it's become possible to operate with smaller addressable pieces of media (chunks/fragments) - rather than traditional segments. It allows earlier movement of media in the streaming pipeline. This solution is based on an on-premises AWS Elemental Live encoder, which we currently don't have. The understanding is - it's quite expensive and possibly we won't afford it :(
Besides ordinary HLS Segment Length setting (BTW we use 1sec as of now), it also provides some interesting settings like:
Chunked Encoding = true
Chunk Length = 200ms
Fragment Length = 1sec
Chunked Transfer = true

There are no such "advanced" settings in AWS MediaLive or in AWS MediaPackage, but we see that:

  1. In AWS MediaLive there are HLS container option for channel output to be set to "fMP4 HLS"
    "Fmp4 hls – Choose this type of container if you want to package the streams (encodes) as fragmented MP4"
    https://docs.aws.amazon.com/medialive/latest/ug/hls-container.html
  2. In AWS MediaPackage there is the following type of endpoint: Common Media Application Format
    https://docs.aws.amazon.com/mediapackage/latest/ug/endpoints-cmaf.html?icmpid=docs_elemental_mediapackage_helppanel

We tried to configure both 1) fMP4 HLS as a container for output to AWS MediaStore and 2) CMAF endpoint for MediaPackage but did not see any difference in resulting end-to-end latency.

The question is:
Does it make sense to continue experimenting with these configuration options to achieve lower latency?

As there are no mentioned above settings in AWS MediaLive and AWS MediaPackage, the feeling is - we tried the wrong way. But it's always better to ask so that I'm asking :)

Thanks for reading and your help in advance,
Igor

Edited by: ibotov on Apr 6, 2021 7:48 AM

ibotov
asked 3 years ago1218 views
8 Answers
0

Hi Igor,

Thank you for asking this question on the AWS Elemental MdeiaLive forums, before we delve to deeply into the issue can you share with us what your current latency is looking like? It would be helpful to see what you are currently seeing as far as your latency and then see if we can shave some time off from that.

Zach

answered 3 years ago
0

Hi Zach,

We managed to achieve 7 seconds latency with HLS.js player. Please refer to the screenshot:

https://www.dropbox.com/s/suc0v44w7431c0b/e2e_latency_medialive.png?dl=0

There are two sides as far as you can see that:

  1. on the left: a stopwatch app - in our case just a bash script rendering current time to be captured, encoded, and upstreamed via RTMP PUSH to AWS Elemental MediaLive input,
  2. on the right: web app with integrated HLS.js player, which plays the media stream from the AWS Cloud (going thru MediaLive -> MediaStore -> CloudFront CDN)

It's important to mention that we forced the player to keep its buffer as tiny as possible - for sure it's not a production case as having this setup we do observe buffering issue quite frequently. In production, we will add few seconds to achieve better stability. But as of now, we are looking for EDGE glass-to-glass Latency - absolutely minimal latency that we can get with cloud-based streaming architecture.

Many thanks,
Igor

ibotov
answered 3 years ago
0

Hi Igor,

Here are a couple areas that you can modify to shave some time off your latency, if you have not already done them, all of these settings will be located under the output video stream settings.

  1. Look ahead Rate Control: Setting it to low will improve latency while reducing output quality for demanding scenes. Low works well mainly if there are no or few scene changes, though.

  2. GOP parameters: It is recommended to use closed GOPs of 1 second duration that can be later on repackaged in 2 seconds segments if needed. Small GOPs without B Frames generally reduce video quality.

  3. B Frames: The more B Frames used in a GOP, the higher the probability that encoding latency will be increased by a few frames for each B Frame added, as the encoding engine will look ahead to the next P Frames to construct the B frame. Using zero B Frames is possible to evacuate this latency impact, but it will require raising the encoding bitrate to keep the same quality level as when using B Frames.

  4. Adaptive Quantization: Turning it off will reduce latency by a few frames. The other Adaptation Quantization options don’t impact latency.

  5. Encoder Buffer Size: The default value is twice the video bitrate in bits, which generates a 2 second delay on the decoder. If set to 1x bitrate, it generates a 1 second delay and slightly impacts the video quality. For a very aggressive low latency target, the buffer size could be set to half of the bitrate, which results in half a GOP (1/2 second) of delay. Of course, the video quality will be more impacted; under 1/2x bitrate, the impact on video quality is more severe.

More information can be found here, it deals with Elemental Live but these settings are also available in MediaLive:

https://aws.amazon.com/blogs/media/how-to-compete-with-broadcast-latency-using-current-adaptive-bitrate-technologies-part-2/

Zach

Edited by: Zacharyb-AWS on Apr 8, 2021 10:16 AM

answered 3 years ago
0

Thanks, Zach for your help. Particularly thanks for the confirmation that encoding tuning in AWS MediaLive makes sense - we tried this and the mentioned 6 seconds latency we have - is in respect of this.

Could you please also advise if it's possible to utilize such a beneficial feature of CMAF with a cloud-based solution (MediaLive) like a "chunked delivery" that the on-premises Elemental Live solution does already provide?

https://aws.amazon.com/blogs/media/part-1-getting-started-with-ultra-low-latency-using-aws-elemental-live/
It explained in a really good way in the article, so let me cite it here:
"How it works
The AWS Elemental Live ULL workflow utilizes Common Media Application Format (CMAF) media files with chunked delivery. The CMAF media files provide smaller addressable fragments of video than traditional HTTP-based file segmentation. These fragments can progress through the delivery chain to the next component before the entire file segment has been created. In traditional HTTP streaming, an entire file had to be completely written before the next component can request it. Other optimizations in the AWS Elemental Live encoder include manifest creation, CDN integration, and player capabilities, mostly based on CMAF and chunk delivery."

Best Regards and Many Thanks,
Igor

ibotov
answered 3 years ago
0

Hi Igor,

We can do Chunked Transfer when using HTTP Live Streaming HLS, it becomes available when you set your CDN Settings to HLS Webdav or HLS Akamai. However it is not supported when delivering to MediaPackage, when using HTTP Live Streaming MediaPackage the optimal settings are included but Chunked Tranfer is not supported by it.

Zach

answered 3 years ago
0

Thanks, Zach for the insights. What you described makes sense to me.

Now I'm just wondering when we set CDN Setting to "HLS media store" (what we currently do in our setup), does media stream transfer work in a Chunked way?

ibotov
answered 3 years ago
0

Hi Igor,

I apologize, I just checked and when using the CDN Setting HLS to MediaStore it does not do chunked transfer.

Zach

answered 3 years ago
0

Thank you very much, Zach, for all your help! Not sure if it's feasible at all with the current MediaLive service architecture and implementation, but an opportunity to reduce streaming latency with chunked transfer in MediaLive to MediaStore scenario - sounds good to me - please consider this as a feature suggestion :) (of course, if it's feasible).

In our current setup 1 sec segment size is used. So the understanding is - potential latency optimization due to potential "chunked transfer" feature utilization might be roughly up to 1 sec for us (potential 17% latency reduction in our case). Not bad :)

Best regards and many thanks,
Igor

ibotov
answered 3 years ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions