Do all EBS volumes have only 2 IO queues?

0

I am deploying SPDK on AWS using EBS volumes, exposing the disks with NVMe over Fabrics.

On SPDK, one can have a variable number of threads to handle the requests and IO. For performance reasons, though, SPDK does not shares the NVMe IO queues between threads, so each thread must have its own queue. This is usually not a problem since often devices have 8+ IO queues.

EBS volumes, however, seem to have only 2 IO queues. This was verified by looking at /sys/class/nvme/nvmeX/queue_count, which is 3 (1 admin queue + 2 IO queues).

This limits SPDK to 2 threads, which is not enough to achieve full performance. For example, using a im4gn.16xlarge instance with 4x local SSDs + 5x EBS volumes, I can get 9.0GB/s write bandwidth when writing directly to the disks (measured with fio). When creating the fabrics devices with the kernel, I can also get 9GB/s (the kernel shares queues between threads). However, when using SPDK, I get 5.3GB/s with one thread and 6.6GB/s with two threads. So clearly more threads (queues) would help.

I tested both gp3 and io2 volume types (including the io2 Block Express types) and they all had 2 IO queues.

So my question is if there is any way to create EBS volume types that have more IO queues.

질문됨 2년 전127회 조회
답변 없음

로그인하지 않았습니다. 로그인해야 답변을 게시할 수 있습니다.

좋은 답변은 질문에 명확하게 답하고 건설적인 피드백을 제공하며 질문자의 전문적인 성장을 장려합니다.

질문 답변하기에 대한 가이드라인

관련 콘텐츠