Why isn't my SC1 or ST1 EBS volume achieving the rated throughput performance?

Short description

When using HDD Amazon EBS volumes, such as SC1 and ST1, keep the following:

• These volumes always use a 1024 KiB I/O token regardless of the actual I/O size used by the workload on the instance. Even if the actual I/O size of the application workload is set to 16 KiB, the volume still uses the entire 1024 KiB size of the I/O token. This wastes most of the token's space. To maximize efficiency, fill the entire 1024KiB.
• When the I/O size of a sequential workload is greater than 32 KiB, Amazon EBS always merges the I/Os into a single I/O operation of 1024 KiB. This merging fills the entire token size.
• When the I/O size is smaller than 32 KiB, or if the workload is random, Amazon EBS doesn't merge the I/Os to 1024 KiB. However, Amazon EBS still uses the entire 1024 KiB token size. This leaves most of the space inside the token empty. Since the I/Os aren't merged, the instance uses more IOPS sending the same amount of data to the volume. This causes a reduction in the burst balance even though the throughput is below its baseline value.

Resolution

To enable your ST1 and SC1 EBS volumes to reach their maximum rated throughput, do the following:

• Set up your application to use an I/O size greater than 32 KiB.
• Verify that your application uses sequential workload.

Example

When calculating throughput, use the following formula:

Throughput = I/Osize * IOPS

If the I/O size is smaller than 32 KiB, the volume hits its IOPS limit, throttling the throughput. When this happens, the volume never achieves it's rated throughput performance.

For example, setting the I/O size to 16 KiB and sending 3 MiB/s of data results in the following:

3MiB/s/16KiB = 192 IOPS

Setting the I/O size to 32 KiB with a sequential/contiguous workload causes Amazon EBS to merge to 1024 KiB. Amazon only sends 3 IOPS, in this case, as shown in the following calculation:

3MiB/s/1024Kib = 3 IOPS

Assume you're using 0.5 TiB (500 GiB) of an ST1 volume. This volume ideally provides a baseline throughput performance of 20 MiB/s and can burst up to 125 MiB/s.

If the volume is bursting at 125 MiB/s, and the applications I/O size is 1024 KiB, then maximum theoretical IOPS equals 125 IOPS.

Throughput / IOsize = 125MiB/s / 1024 KiB = 125 IOPS.

However, if the application uses an I/O size of 16 KiB, then sending 3 MiB of data uses 192 IOPS. The application can't push 192 IOPS because the volume only achieves a theoretical maximum of 125 IOPS. In this case, the volume throttles IOPS to 125. In this scenario, the actual throughput is as follows:

Actual throughput = 16 KiB * 125 = 1.95 MiB/s

As shown in the preceding calculations, IOPS is throttled to 125 IOPS, so Amazon EBS throttles the throughput to 1.95 MiB/s. This throttling occurs even though the burstable theoretical throughput for the volume is 125 IOPS given that the application used 1024 KiB I/O size.

Related information

I/O characteristics and monitoring

Amazon EBS volume types