Instance store performance

The performance you're experiencing with EC2 instance store volumes is indeed lower than what these instances are capable of. There are a few factors to consider and steps you can take to improve performance:

1. Instance Store Initialization: Some instance store volumes suffer from a first-write penalty unless initialized. This is particularly true for NVMe SSD volumes. To optimize disk performance, you should initialize the volumes before running your tests. This can be done by writing once to every drive location before production use.

2. Expected Performance: Based on the specifications provided by AWS, the instance types you mentioned should be capable of much higher performance:

   • i4g.xlarge: Up to 125,000 IOPS for random read operations and 100,000 IOPS for write operations
   • m6id.xlarge: Up to 87,500 IOPS for random read operations and 35,000 IOPS for write operations
   • m5ad.2xlarge: Up to 70,000 IOPS for random read operations and 40,000 IOPS for write operations

3. Testing Methodology: The method you're using (dd) is good for basic testing, but it might not fully utilize the capabilities of NVMe SSDs. Consider using more sophisticated benchmarking tools that can generate multiple threads and queue depths to fully stress the storage.

4. TRIM Support: All the instance types you mentioned support TRIM, which can help maintain SSD performance over time. Ensure that TRIM is enabled on your file system.

5. File System Choice: While ext4 is a good general-purpose file system, you might want to experiment with other file systems optimized for SSDs, such as XFS or F2FS, to see if they provide better performance for your specific workload.

6. Instance Size: Larger instances often have better I/O performance. If possible, test with larger instance sizes to see if performance scales as expected.

7. Queue Depth and I/O Size: NVMe SSDs often perform best with higher queue depths and larger I/O sizes. Adjust these parameters in your benchmarking tools to find the optimal configuration.

To improve performance:

1. Initialize your volumes before testing.
2. Use benchmarking tools designed for NVMe SSDs, such as fio or nvme-cli.
3. Experiment with different file systems and mount options.
4. Ensure your kernel and NVMe drivers are up to date.
5. Consider using larger instance sizes if your workload requires higher performance.

By addressing these factors, you should be able to achieve performance levels much closer to the specifications provided by AWS for these instance types.
Sources
Specifications for Amazon EC2 accelerated computing instances - Amazon EC2
Specifications for Amazon EC2 storage optimized instances - Amazon EC2
Specifications for Amazon EC2 general purpose instances - Amazon EC2
Initialize instance store volumes on EC2 instances - Amazon Elastic Compute Cloud