HI @DaneLLL,
we used 35.2.1 on Xavier NX with Intel I210, and we found the throughtput of I210 is very slow, only around 10.3 Mbits/sec…
On the other hand, the 32.7.1 did not have this kind of issue on I210.
The steps I did as following:
- “iperf3 -s” for a Notebook
- on the other hand, on xavier nx, I did the command: iperf3 -c “192.168.100.102” -t 120 -i 10 -P 4
Could you please help to check it? Thanks a lot.
dmesg.txt (101.8 KB)
iperf.txt (6.5 KB)
log.7z (127.6 KB)
kernel_tegra194-p3668-0001-p3509-0000.dtb (316.3 KB)
Hi,
Could you try r35.3.1(Jetpack 5.1.1)? Would be great if you can upgrade to latest release and give it a try.
Hi @DaneLLL ,
we tried r35.3.1 and it has the same issue of lower throughput on Intel I210.
The logs are attached:
dmesg.txt (104.1 KB)
iperf.txt (6.5 KB)
kernel_tegra194-p3668-0001-p3509-0000.dtb (319.0 KB)
log.7z (133.6 KB)
Hi,
There is PCIe slot on Xavier developer kit so we try on Xavier developer kit + i210 adapter The result is identical on Jetpack 4.6.1 and d5.1.1:
nvidia@tegra-ubuntu:~$ iperf3 -s
-----------------------------------------------------------
Server listening on 5201
-----------------------------------------------------------
Accepted connection from 10.0.0.1, port 50652
[ 5] local 10.0.0.2 port 5201 connected to 10.0.0.1 port 50654
[ ID] Interval Transfer Bandwidth
[ 5] 0.00-1.00 sec 108 MBytes 904 Mbits/sec
[ 5] 1.00-2.00 sec 112 MBytes 941 Mbits/sec
[ 5] 2.00-3.00 sec 112 MBytes 941 Mbits/sec
[ 5] 3.00-4.00 sec 112 MBytes 941 Mbits/sec
nvidia@tegra-ubuntu:~$ iperf3 -c 10.0.0.2 -b 0 -l 16K -t 60 -i 1
Connecting to host 10.0.0.2, port 5201
[ 4] local 10.0.0.1 port 50654 connected to 10.0.0.2 port 5201
[ ID] Interval Transfer Bandwidth Retr Cwnd
[ 4] 0.00-1.00 sec 114 MBytes 952 Mbits/sec 0 402 KBytes
[ 4] 1.00-2.00 sec 112 MBytes 941 Mbits/sec 0 402 KBytes
[ 4] 2.00-3.00 sec 112 MBytes 941 Mbits/sec 0 402 KBytes
[ 4] 3.00-4.00 sec 112 MBytes 942 Mbits/sec 0 424 KBytes
We would expect PCIe throughput is identical for Xavier and Xavier NX, but this looks specific to using Xavier NX. Could you run sudo tegrastats while runnig the test on Jetpack 4.6.1 and 5.1.1? See if there is any clue from the system status.
Hi @DaneLLL ,
I think you should use a notebook with Linux OS or Windows OS to be a iperf server, and Xavier developer kit to be a iperf client. We have tried three different notebooks, and we found the same result: lower throughput on I210.
I will run sudo tegrastats while runnig the test and update the information to you later.
Thank you a lot.
Additionally, the issue occurs when I210 to be a client. If I210 to be a server, there might be no problem.
Hi,
We see similar result when PC is a server and I210 is a client:
davey@dave-yu-nvidia-pc:/mnt/1T-HDD/BSP/r35.3.1/Linux_for_Tegra$ iperf3 -s
-----------------------------------------------------------
Server listening on 5201
-----------------------------------------------------------
Accepted connection from 10.0.0.2, port 38558
[ 5] local 10.0.0.1 port 5201 connected to 10.0.0.2 port 38560
[ ID] Interval Transfer Bandwidth
[ 5] 0.00-1.00 sec 112 MBytes 941 Mbits/sec
[ 5] 1.00-2.00 sec 112 MBytes 941 Mbits/sec
[ 5] 2.00-3.00 sec 112 MBytes 941 Mbits/sec
[ 5] 3.00-4.00 sec 111 MBytes 934 Mbits/sec
[ 5] 4.00-5.00 sec 112 MBytes 941 Mbits/sec
nvidia@tegra-ubuntu:~$ iperf3 -c 10.0.0.1 -b 0 -l 16K -t 60 -i 1
Connecting to host 10.0.0.1, port 5201
[ 5] local 10.0.0.2 port 38560 connected to 10.0.0.1 port 5201
[ ID] Interval Transfer Bitrate Retr Cwnd
[ 5] 0.00-1.00 sec 114 MBytes 953 Mbits/sec 0 375 KBytes
[ 5] 1.00-2.00 sec 112 MBytes 941 Mbits/sec 0 375 KBytes
[ 5] 2.00-3.00 sec 112 MBytes 944 Mbits/sec 0 434 KBytes
[ 5] 3.00-4.00 sec 112 MBytes 937 Mbits/sec 0 536 KBytes
[ 5] 4.00-5.00 sec 112 MBytes 941 Mbits/sec 0 536 KBytes
So for now we don’t observe the issue on AGX Xavier developer kit. It may be specific to using Xavier NX. Have you tried sudo tegrastats? Not sure if you have run it and check if anything suspicous from tne prints.
Hi @DaneLLL ,
The following attached files are those we used sudo tegrastats during iperf3 on 5.1.1 and 4.6.1.
iperf_4.6.1.txt (6.5 KB)
logs_4.6.1.7z (79.6 KB)
tegrastats_4.6.1.txt (33.9 KB)
iperf_5.1.1.txt (6.5 KB)
logs_5.1.1.7z (164.4 KB)
tegrastats_5.1.1.txt (41.9 KB)
Thanks a lot.
Hi,
The system has more idle time on Jetpack 5.1.1. Somehow it does not run as busy as 4.6.1. It seems like device tree is not well configured. Could you share the dts file for reference? And do you connect I210 to pcie@14160000 or pcie@141a0000?
Hi @DaneLLL ,
The following attached are from my Xavier NX with Jetpack5.1.1. The I210 is connected to pcie@141a0000.
Thank you.
dmesg.txt (97.3 KB)
lshw.txt (11.6 KB)
lspci.txt (455 Bytes)
lsusb.txt (547 Bytes)
kernel_tegra194-p3668-0001-p3509-0000.dtb (319.0 KB)
target.dts (393.5 KB)
Hi,
For information, you have mentioned PCIe switch in
[35.3.1] Intel I210 throughput issue on Jetson Orin Nano Devkit - #11 by fengying_chu
The topic is for Orin Nano, but you have same design for Xavier NX like:
[35.3.1] Intel I210 throughput issue on Jetson Orin Nano Devkit - #4 by fengying_chu
There are two I210 connected to pcie@141a0000. Is this correct?
Hi @DaneLLL ,
Yes, two I210 are connected to pcie@141a0000 on Xavier NX on Jetpack 5.1.1.
In addition to Xavier NX, we also found that Orin Nano and Orin NX both have the same issue on I210 with PCIe switch on Jetpack5.1.1.
The I210 with PCIe switch hardware we used is as the following pictures:
您好，我們使用的I210如下圖，用此I210 with PCIe switch在Xavier NX, Orin Nano and Orin NX的Jetpack 5.1.1皆有throughput低的問題。
然而使用Jetpack 4.6.1在Xavier NX沒有此問題。
感謝。
Hi @DaneLLL ,
we have no problem on Intel I210 without PCIe switch.
This issue might come from I210 with PCIe switch.
Please let us know if Nvidia does not have a plan to correct it. Thank you very much.
Hi,
We are checking it with our team. Will update if there is conclusion about the support.
Hi,
Have checked with our team and we would suggest connect to I210 directly. It is not validated with PCIe switch so certain use-case may not work as expected. The device tree looks fine so probably some configs have to be enabled when there is a switch. This would need other users to share experience.
