[QST] Use which among NVLink/NVSwitch/NVC2CLink/NVSciPcie to chain 2 DriveOrin Devices to scale performance?

[lizhensheng]

Required Info:

• Software Version
  DRIVE OS 6.0.6
• Target OS
  Linux
• SDK Manager Version
  1.9.2.10884
• Host Machine Version
  native Ubuntu Linux 20.04 Host installed with DRIVE OS DOCKER Containers

Describe the bug

There are several technologies about interconnect between NVIDIA GPUs. Including NVLink What Is NVLink? | NVIDIA Blogs , NVSwitch NVSwitch: Leveraging NVLink to Maximum Effect | NVIDIA Technical Blog , NVC2CLink NVIDIA Opens NVLink for Custom Silicon Integration | NVIDIA Newsroom and so on.

1. From the website I see there is Ampere support with 3rd-NVLink, since DriveOrin is with Ampere, Does DriveOrin support NVLink so we can chain 2 Orin Devices to scale the performance? If so, how to get the hardware and software guidance?
2. Does DriveOrin support NVSwitch/NVC2CLink? If so, how to get the hardware and software guidance?
3. What’t the platform solution to chain 2 Orin Devices to transfer workload effieiently, is it NVSCI over PCIe?
4. If so, Does NVSciPcie support unified GPU addressing like what NVLink does?

Additional context

I’ve search and read several topics in NVIDIA Forum in the following, I’d like to get official support in forum specifically with DriveOrin

Jetson AGX Xavier NVLINK

I am designing a custom carrier PCBA that will incorporate multiple AGX Xavier modules. I would like to utilize NVLINK to directly connect two of the AGX modules. However, I have some questions regarding NVLINK. Do you have any reference documentation or app notes regarding the NVLINK interface? Are all eight lanes from PCIE controller C5 required for NVLINK to function properly, or can only 4 lanes from PCIE controller C5 be routed between two AGX modules for NVLINK data transmission (at…

Can we connect multiple Jetson AGX Xavier with NVLink

Hello, We want to build a system that composed of a host CPU and some Jetson AGX-Xavier (> 3). The idea would be that the CPU will distribute the computatation as well as data acros (using MPI for example) these AGX-Xavier. I have read in one topic on “devtalk.nvidia” that NVLink is not supported on AGX-Xavier (https://devtalk.nvidia.com/default/topic/1044059/jetson-agx-xavier/can-we-chain-multiple-xavier-jetsons-together-either-in-ep-mode-or-via-ntb-/). Does AGX-Xavier supports NVLink now?

Can we chain multiple Xavier Jetson's together either in EP mode or via NTB?

We have a POC we are building using our Gen4 PCIe Switch with Multiple Jetson Xavier’s. The idea would be to showcase multiple example visual flows using each Jetson and ideally include some “host to host” communication. I was wondering if there is any information on setting up either of these use cases: CASE 1: Configure both Jetson’s for EP mode so that they both show up under a third party host (x8 gen4 EP) Are the GPUs accessible this way? CASE 2: Configure both Jetson’s over NTB in …

How another CPU communicate with Xavier through PCIE? (Solved)

And under windows device manager what will it show as the device ? If your question is about the class of the device, then, it is shown as a device of type ‘Memory Controller’ class Is there a way to flash the xavier as endpoint through the Jetpack SDK Manager? Please refer to https://devtalk.nvidia.com/default/topic/1048582/jetson-agx-xavier/the-problem-about-xavier-pcie-endpoint-mode/ post What does it mean ‘compile tegra-pcie-ep-mem.c on host PC’ how exactly do I do that is ther…

[lizhensheng]

Friendly ping @SivaRamaKrishnaNV @VickNV

[VickNV]

NVLink isn’t support on the devkit (Orin-X has no NVLink interface).

@lizhensheng

4. If so, Does NVSciPcie support unified GPU addressing like what NVLink does?

Please elaborate on the unified addressing you are asking for. Please refer to the following C2C communication:

• Chip-to-Chip Communication | NVIDIA Docs

[lizhensheng]

@VickNV

Please elaborate on the unified addressing you are asking for.

FYI, the unified addressing I am looking for is in here CUDA C++ Programming Guide (nvidia.com)

Depending on the system properties, specifically the PCIe and/or NVLINK topology, devices are able to address each other’s memory (i.e., a kernel executing on one device can dereference a pointer to the memory of the other device). This peer-to-peer memory access feature is supported between two devices if cudaDeviceCanAccessPeer() returns true for these two devices.