Do all EBS volumes have only 2 IO queues?

0

I am deploying SPDK on AWS using EBS volumes, exposing the disks with NVMe over Fabrics.

On SPDK, one can have a variable number of threads to handle the requests and IO. For performance reasons, though, SPDK does not shares the NVMe IO queues between threads, so each thread must have its own queue. This is usually not a problem since often devices have 8+ IO queues.

EBS volumes, however, seem to have only 2 IO queues. This was verified by looking at /sys/class/nvme/nvmeX/queue_count, which is 3 (1 admin queue + 2 IO queues).

This limits SPDK to 2 threads, which is not enough to achieve full performance. For example, using a im4gn.16xlarge instance with 4x local SSDs + 5x EBS volumes, I can get 9.0GB/s write bandwidth when writing directly to the disks (measured with fio). When creating the fabrics devices with the kernel, I can also get 9GB/s (the kernel shares queues between threads). However, when using SPDK, I get 5.3GB/s with one thread and 6.6GB/s with two threads. So clearly more threads (queues) would help.

I tested both gp3 and io2 volume types (including the io2 Block Express types) and they all had 2 IO queues.

So my question is if there is any way to create EBS volume types that have more IO queues.

已提問 2 年前檢視次數 127 次
沒有答案

您尚未登入。 登入 去張貼答案。

一個好的回答可以清楚地回答問題並提供建設性的意見回饋,同時有助於提問者的專業成長。

回答問題指南