Jetson Orin AGX - How to use GPUDirect RDMA with Deepstream or Tensorrt

Robotics & Edge Computing Jetson & Embedded Systems Jetson AGX Orin
,
1

I am working on creating a driver for a PCIe frame grabber for the Orin AGX. I have reviewed the sample driver for RDMA here:

It shows how to use RDMA with cudaHostAlloc calls. However, my end users use higher level SDKs for running their models. I want to understand how to integrate RDMA with user applications run with Tensorrt and user applications run using Deepstream. Can someone explain this or point me to documentation that can help me understand how to access this memory with the higher level SDKs?

5

Hi,

TensorRT and Deepstream can use the buffer created by cudaHostAlloc.
When you get the GPU buffer pointer, please feed it into the library accordingly and run the high-level SDK.

For example:
Update the cudaMalloc to use RDMA buffer:

Thanks.

7

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.