How many DRAM channels are needed to saturate the PCIe bandwidth on F1?

0

Hello,

I'm an image processing researcher and I am interested in using Vitis on the F1 to accelerate some applications. All of these applications will read in streams of 1920 x 1080 images and write out 1920 x 1080 images. The total amount of data read will be the same as the total amount of data written, and the images will be loaded in raster-scan order (one row at a time). The applications are easy to parallelize and can be fully pipelined, so that in their steady state they can accept new data every cycle and write finished data every cycle.

My understanding is that the connection between the FPGA and its memory is PCIe 3.0 x16, and that the peak bidirectional bandwidth is approximately 32 GB / sec (16 GB / sec of writes and reads respectively).

From reading other questions / documentation I gather that there are four DRAM channels, each with a peak bandwidth of 17 GB / sec (https://forums.aws.amazon.com/thread.jspa?threadID=269319).

Am I understanding the PCI and DRAM bandwidth correctly? And if so can a Vitis application reach the peak bandwidth of the PCIe connection by using only 2 of the 4 DRAM channels (one for reads and one for writes).

Thanks in advance!

asked 4 years ago326 views
3 Answers
0
Accepted Answer

Hi,

Yes, F1 Instances have PCIe gen3 x16 Link between the FPGA and the Host, with FPGA as PCIe endpoint. So theoretically PCIe link should supports max bandwidth of 16GB/s per direction. However due to system latencies and other overheads, the design may not be able to hit the theoretical max. In F1, max achievable bandwidth is also dependent on the Custom Design (CL) as well.

You will still be able to achieve max bandwidth using only one DRAM channel. Following are few other posts that may help you:
https://forums.aws.amazon.com/thread.jspa?messageID=818846&#818846
https://forums.aws.amazon.com/thread.jspa?messageID=812716&#812716
https://forums.aws.amazon.com/thread.jspa?messageID=940247&#940247
https://forums.aws.amazon.com/thread.jspa?messageID=897261&#897261
https://forums.aws.amazon.com/thread.jspa?messageID=846147&#846147

Please contact us if you need any additional details.

Thanks!
Chakra

AWS
answered 4 years ago
0

Thanks Chakra!

You mentioned that it will be possible to reach peak bandwidth with only 1 DRAM channel. So is it correct to say that the unidirectional bandwidth of one DRAM channel is ~17 GB/sec, and that each channel supports 17 GB/sec per direction?

answered 4 years ago
0

Hello,

The DDR data pins are shared between read and write accesses. Therefore, the max available DDR bandwidth of ~17GB/s is shared between reads and writes. You can potentially achieve maximum performance when your accesses are either write-only or read-only.

Please let us know if you seek additional details.

Thanks!
Chakra

AWS
answered 4 years ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions