/dev/xdma* not seen after DDR_A included in the design.

Question

Hello all,  
  
I have my test design, where I include DDR_C and a single BRAM in the CL, and connect them, via AXI Interconnect to the M_AXI_PCIS port of the F1 Inst. This design can be seen in `1.png` attachment. This design is functional - tested, and works as expected.  
However, when I try to include DDR_A in the same design (attachment `2.png`), and I scan the PCIe devices by doing `ls /dev/xdma*`, I don't observe any devices! Usually I'd expect to have four h2c channels, four c2h channels, events objects etc.  
  
What am I doing wrong with this design?  
  
P.S. Of course, all peripherals are properly given addresses, implementation finished and AGFI was properly created and loaded.  
  
Edited by: jelicicm on Jan 3, 2021 11:49 PM

Answer

Hi jelicicm,  
  
We see xrt in the log that you sent, wanted to know if you are using the Vitis flow?  
  
Thanks,  
Sunil.

Answer

Hi jelicicm,  
  
On the FPGA Developer AMI(Centos), XRT is pre-installed and starting mpd loads a Default AFI that does not instantiate DDR's. XRT:   is used for Vitis/SDAccel based accelerators.   
  
In your case since you are creating an accelerator using HLx, please remove XRT first and install the XDMA driver provided in the aws-fpga repository.  
  
Removing XRT:

```
sudo yum remove xrt xrt-aws
sudo rmmod xocl
```

XDMA installation instructions are available here:   
  
Let us know if this doesn't work for you and we'd be happy to help.  
  
-Deep

Answer

Hello AWSDeep, and thank you for your reply.   
Yes, this did help me solve my problems, but it wasn't enough. For some reason, aws::make_ipi set DEVICE_ID parameter of F1 instance to 0xf010, so when I manually changed this to 0xf000 (found this value while browsing older, working examples) it started showing all ports in `/dev/xdma*`  
  
Thank you for your help awsdeep and awssunil!

Answer

Hello AWSSunil, thanks for taking the interest in this problem.   
EDIT: I don't know why the Forum added http:// before all dmesg outputs, of course I didn't mean to do that.  
EDIT2: The Forum keeps chaning my original logs and adding links to them. Here is the original message on pastebin, I hope that this is OK.  
https://pastebin.com/CCtunewf  
  
Edited by: jelicicm on Jan 5, 2021 1:16 PM  
  
Edited by: jelicicm on Jan 5, 2021 1:17 PM

Answer

Hi AWSSunil,  
  
Do you mean "am I using HLx"? In that case - yes. I wanted to go without HLx, but then it seems that my Vivado doesn't know `aws::make_ipi` command that I need to instantiate AWS IP.  
  
the only appearance of XRT is because I've run `xbutil scan`, because I was hoping it might help me find a solution, or a cause of this problem. This is the environment that I'm using:  
Ubuntu 20.04, Vivado 2019.1 64-bit, AWS-FPGA latest commit from master branch,  when I run `hdk_setup.py` I get the info that patch AR73068 gets applied, shell_v04261818 is used, and there is a Vivado_init.tcl script that gets executed when I startup my Vivado that sources  $::env(HDK_SHELL_DIR)/hlx/hlx_setup.tcl  
  
On AWS I have CentOS (I believe its 7, not in the position to check right now), and a pretty old kernel (3.x.y.)

Answer

Hi jelicicm  
  
After you include the DDR_A, can you please provide the pci device id after the afi is loaded and also can you provide us the syslog/dmesg log output?  
  
Thanks,  
Sunil.

/dev/xdma* not seen after DDR_A included in the design.

Relevant content