Transferring data using PCIe between Orin and Xavier

[shrinidhi.ramasubramanian]

I have connected an Jetson Orin AGX (endpoint) and Xavier using PCIe (root).

I followed the instructions in PCIe Endpoint Mode — Jetson Linux Developer Guide documentation

And I also increased the size on BAR0 in nvidia/drivers/pci/endpoint/functions/pci-epf-nv-test.c in the endpoint to 128M.

I see on the root that the endpoint is enumerated correctly, and I see the 128MB non-prefetchable memory.

I was successfully able to send across a “Hello World!” from endpoint using mmap() and memcpy() functions, and I was able to read it back as well.

However, I tried sending a 20KB text file from endpoint, and I could read back only 4KB on the root (the rest of the data was just garbage). I was wondering how to debug this.

Any help would be really appreciated.

Thanks!

[carolyuu]

Hi,
Here are some suggestions for the common issues:

1. Performance

Please run the below command before benchmarking deep learning use case:

$ sudo nvpmodel -m 0
$ sudo jetson_clocks

2. Installation

Installation guide of deep learning frameworks on Jetson:

• TensorFlow: Installing TensorFlow for Jetson Platform - NVIDIA Docs
• PyTorch: Installing PyTorch for Jetson Platform - NVIDIA Docs
  We also have containers that have frameworks preinstalled:
  Data Science, Machine Learning, AI, HPC Containers | NVIDIA NGC

3. Tutorial

Startup deep learning tutorial:

• Jetson-inference: Hello AI World guide to deploying deep-learning inference networks and deep vision primitives with TensorRT and NVIDIA Jetson
• TensorRT sample: Jetson/L4T/TRT Customized Example - eLinux.org

4. Report issue

If these suggestions don’t help and you want to report an issue to us, please attach the model, command/step, and the customized app (if any) with us to reproduce locally.

Thanks!

[shrinidhi.ramasubramanian]

@carolyuu your response is not related to PCIe :)

[WayneWWW]

please check

https://forums.developer.nvidia.com/t/xavier-in-ep-mode-shared-memory-the-default-size-is-size-64k-but-it-can-only-be-used-4kb/295797/6

[shrinidhi.ramasubramanian]

@WayneWWW Thank you for your response!

So in this case, should I just do a memcpy() our data into the output we get for pci_epf_nv_test.0: BAR0 RAM virt or the pci_epf_nv_test.0: BAR0 RAM IOVA:?

(edit: Neither worked)