Question about Xavier NX pcie endpoint virtual ethernet bandwidth

Autonomous Machines Jetson & Embedded Systems Jetson Xavier NX
, ,
#1

Hello guys,

I made pcie board with Xavier NX for transfer camera image data on nx to host PC.

image
image1115×659 20 KB

Xavier NX is set to EP mode, it connect to host PC (RP) through virtual ethernet using pcie port.

Connection is well functionning and now I can transfer camera image data on nx to host PC.

I have 4 cameras to transfer image data, each camera image data’s bandwidth is 1.3Gbps total 5.2Gbps camera image data is transfered with gstreamer udp stream.

Gstreamer udp stream is stable without any data loss.

So I can transfer total 5.2Gbps nx camera image data to host pc with gstreamer udp stream now.

It tested on jetpack 4.4.1 and 4.6 with set cpu performace to max on both side.

I have some question about pcie EP virtual ehternet bandwidth.

  1. Virtual Ethernet Performance for Jetson Xavier NX - #8 by WayneWWW in this topic, nvidia moderator says that his got around 5Gbps with EP interface.
    And he says that his test enviroments is Link width 8x, Link speed: Gen4.
    But Xavier NX’s soc has only 4x pcie data line, how can he test about that?
    If I use only 4x pcie data line, can I get around 5Gbps? or half?

  2. According to question 1, if it can get only half of 5Gbps, how can I transfer 5.2Gbps total camera image data with gstreamer udp stream? is it possible?

  3. I tested bandwidth using iperf3, but result is odd.
    It is slow in one direction.
    Test is right? Why one direction is slow than other?

[ Case 1, use TCP ]

[ Case 2, use UDP ]

  1. According to question 3, from xavier nx to host pc bandwidth is slower than 5Gbps.
    How can I transfer 5.2Gbps total camera image data with gstreamer udp stream?
    iperf3 is wrong? Which app is good to test the exact bandwidth?
    And, performance is upgraded since that topic? Virtual Ethernet Performance for Jetson Xavier NX - #8 by WayneWWW

I wonder about the exact EP virtual ethernet bandwidth and performance.
I confused that if bandwidth is under 5Gbps, how can I transfer 5.2Gbps data.
And I also confused why tested banwith is slower than nvidia moderator says.

If I know wrong or tested wrong, answer me my question. or not.

Thank you guys.

#3

The claimed bandwidth is limited by the software framework and not by the PCIe link width/speed. So, even Gen-4, x4 link would give the same bandwidth as that of Gen-4, x8 link.

As far as the change in bandwidth in one direction is considered, did you try maximizing the Jetson performance (through nvpmodel command) first before running jetson clocks? Also, please tegrastats to see if the CPU is indeed clocked at max frequencies. Ref:- https://docs.nvidia.com/jetson/archives/l4t-archived/l4t-325/index.html#page/Tegra%20Linux%20Driver%20Package%20Development%20Guide/power_management_jetson_xavier.html#wwpID0E0YB0HA

#4

Yes.
I tried maximizing the jetson performance using jetson clocks command, but
bandwidth is still under 5Gbps.
Could you answer the question 4?
I successed to transfer 5.2Gbps data to host pc perfectly even bandwidth is under 5Gbps.
So I confused why can do that.
Using iperf3 is wrong way to check the bandwidth?

Thanks.