Trying RDMA on Jetson Xavier

Infrastructure & Networking Software And Drivers RDMA Software For GPU
, ,
1

Hello NVIDIA Community,

I’m working on prototyping GPUDirect RDMA on a Jetson Xavier device and am seeking guidance on testing procedures and hardware requirements. I’ve looked into the NVIDIA jetson-rdma-picoevb repository, but it doesn’t clearly outline the hardware and software dependencies needed.

Current Setup:

  • Software: Mellanox OFED is installed.
  • Issue: Running ibv_devinfo or RDMA examples results in “No IB devices found.” Firmware updates also indicate no devices found.

Questions:

  1. Hardware Requirement: Do I need to purchase and connect a Mellanox ConnectX NIC to the Jetson Xavier to run basic RDMA examples like rc_pingpong?
  2. PCIe RDMA Testing: Can I test PCIe-based RDMA between two Jetson devices without a ConnectX NIC?
  3. Resources: Are there any workflow/setup instructions available on the NVIDIA portal or elsewhere that detail the necessary steps for setting up GPUDirect RDMA on Jetson Xavier?
  4. Does Jetson Xavier supports both RoCE and PCIe based RDMA?

Any guidance or resources you could provide would be greatly appreciated.

Thank you!