I have a custom-designed carrier board for Xavier NX. I use iperf to send UDP packages to test the ethernet speed and the speed of gigabit ethernet only has 478 Mbits/sec on JetPack 5.0.2 but the speed can reach 1000 Mbits/sec on JetPack 4.6.1. Should I modify DTS or other settings?
nvidia@tegra-ubuntu:~/phoronix-test-suite$ phoronix-test-suite run pts/iperf
No Internet Connectivity
iPerf 3.7:
pts/iperf-1.1.1
Network Test Configuration
Server Address [Use 'localhost' if wishing to benchmark the local system/server performance.]
Enter Value: 192.168.100.100
Server Port [The default iperf3 server port is 5201.]
Enter Positive Number: 12345
1: 10 Seconds
2: 30 Seconds
3: 60 Seconds
4: 360 Seconds
5: Test All Options
** Multiple items can be selected, delimit by a comma. **
Duration: 1
1: TCP
2: UDP
3: UDP - 100Mbit Objective
4: UDP - 1000Mbit Objective
5: Test All Options
** Multiple items can be selected, delimit by a comma. **
Test: 3,4
1: 1
2: 5
3: 10
4: 20
5: 32
6: 64
7: Test All Options
** Multiple items can be selected, delimit by a comma. **
Parallel: 1
Phoronix Test Suite v10.8.4
System Information
PROCESSOR: ARMv8 rev 0 @ 1.91GHz
Core Count: 4
Cache Size: 4 MB
Scaling Driver: tegra194 schedutil
GRAPHICS: NVIDIA TEGRA
Vulkan: 1.3.212
Screen: 4096x2160
MOTHERBOARD: EDK II 1.0-d7fb19b
BIOS Version: 1.0-d7fb19b
Network: Intel Device 0d9f
MEMORY: 16GB
DISK: 16GB DG4016
File-System: ext4
Mount Options: relatime rw
Disk Details: Block Size: 4096
OPERATING SYSTEM: Ubuntu 20.04
Kernel: 5.10.104 (aarch64)
Desktop: GNOME Shell 3.36.9
Display Server: X Server 1.20.13
Compiler: GCC 9.4.0
Security: itlb_multihit: Not affected
+ l1tf: Not affected
+ mds: Not affected
+ meltdown: Not affected
+ spec_store_bypass: Not affected
+ spectre_v1: Mitigation of __user pointer sanitization
+ spectre_v2: Mitigation of Branch predictor hardening
+ srbds: Not affected
+ tsx_async_abort: Not affected
Would you like to save these test results (Y/n): n
iPerf 3.7:
pts/iperf-1.1.1 [Server Address: 192.168.100.100 - Server Port: 12345 - Duration: 10 Seconds - Test: UDP - 100Mbit Objective - Parallel: 1]
Test 1 of 2
Estimated Trial Run Count: 3
Estimated Test Run-Time: 3 Minutes
Estimated Time To Completion: 5 Minutes [09:20 UTC]
Started Run 1 @ 09:16:13
Started Run 2 @ 09:16:30
Started Run 3 @ 09:16:48
Server Address: 192.168.100.100 - Server Port: 12345 - Duration: 10 Seconds - Test: UDP - 100Mbit Objective - Parallel: 1:
100
100
100
Average: 100 Mbits/sec
Deviation: 0.00%
iPerf 3.7:
pts/iperf-1.1.1 [Server Address: 192.168.100.100 - Server Port: 12345 - Duration: 10 Seconds - Test: UDP - 1000Mbit Objective - Parallel: 1]
Test 2 of 2
Estimated Trial Run Count: 3
Estimated Time To Completion: 1 Minute [09:17 UTC]
Started Run 1 @ 09:17:11
Started Run 2 @ 09:17:28
Started Run 3 @ 09:17:45
Server Address: 192.168.100.100 - Server Port: 12345 - Duration: 10 Seconds - Test: UDP - 1000Mbit Objective - Parallel: 1:
456
474
472
Average: 467 Mbits/sec
Deviation: 2.11%
I use rgmii. I also tried the original L4T 35.1 on Xavier NX devkit and it has same issue. Following is the dts i used. tegra194-p3668-0001-p3509-0000.dts (390.4 KB)
Does it has any different with your dts?
Hey you two. I’ve just made a test after I’ve seen this thread and can confirm about 500Mbit/sec transfer speed on Jetpack 5.0.2. I did not make any comparison to older versions though.
My system is highly modified though. I’d be interested in finding what causes this tough.
What is your test environment and test command (tcp/udp)? Ethernet test is very sensitive to those configuration and setup. Thus need to know this info first.
For example, are you doing this test with NX-NX connection or you connect NX to other switch/hub?
NX-NX is the most precise way to test this.
I used sdkmanager to flash Jetpack 5.0.2 and use NV devkit to test it. I connect Xavier NX to the gigabit ethernet of my PC directly and there are not are not any ethernet switch and router between them. The ethernet speed still is about 500 Mbits/sec. But it never happen on JetPack 4.6.1. Do you have any suggestions?
nvidia@ubuntu:~/phoronix-test-suite$ phoronix-test-suite run pts/iperf
No Internet Connectivity
iPerf 3.7:
pts/iperf-1.1.1
Network Test Configuration
Server Address [Use 'localhost' if wishing to benchmark the local system/server performance.]
Enter Value: 192.168.100.100
Server Port [The default iperf3 server port is 5201.]
Enter Positive Number: 12345
1: 10 Seconds
2: 30 Seconds
3: 60 Seconds
4: 360 Seconds
5: Test All Options
** Multiple items can be selected, delimit by a comma. **
Duration: 1
1: TCP
2: UDP
3: UDP - 100Mbit Objective
4: UDP - 1000Mbit Objective
5: Test All Options
** Multiple items can be selected, delimit by a comma. **
Test: 3,4
1: 1
2: 5
3: 10
4: 20
5: 32
6: 64
7: Test All Options
** Multiple items can be selected, delimit by a comma. **
Parallel: 1
Phoronix Test Suite v10.8.4
System Information
PROCESSOR: ARMv8 rev 0 @ 1.91GHz
Core Count: 4
Cache Size: 4 MB
Scaling Driver: tegra194 schedutil
GRAPHICS: NVIDIA TEGRA
Vulkan: 1.3.212
Screen: 4096x2160
MOTHERBOARD: EDK II 1.0-d7fb19b
BIOS Version: 1.0-d7fb19b
Network: Realtek RTL8822CE 802.11ac PCIe
MEMORY: 16GB
DISK: 16GB DG4016
File-System: ext4
Mount Options: relatime rw
Disk Details: Block Size: 4096
OPERATING SYSTEM: Ubuntu 20.04
Kernel: 5.10.104-tegra (aarch64)
Desktop: GNOME Shell 3.36.9
Display Server: X Server 1.20.13
Compiler: GCC 9.4.0
Security: itlb_multihit: Not affected
+ l1tf: Not affected
+ mds: Not affected
+ meltdown: Not affected
+ spec_store_bypass: Not affected
+ spectre_v1: Mitigation of __user pointer sanitization
+ spectre_v2: Mitigation of Branch predictor hardening
+ srbds: Not affected
+ tsx_async_abort: Not affected
Would you like to save these test results (Y/n): n
iPerf 3.7:
pts/iperf-1.1.1 [Server Address: 192.168.100.100 - Server Port: 12345 - Duration: 10 Seconds - Test: UDP - 100Mbit Objective - Parallel: 1]
Test 1 of 2
Estimated Trial Run Count: 3
Estimated Test Run-Time: 3 Minutes
Estimated Time To Completion: 5 Minutes [20:09 CST]
Started Run 1 @ 20:05:42
Started Run 2 @ 20:05:59
Started Run 3 @ 20:06:16
Server Address: 192.168.100.100 - Server Port: 12345 - Duration: 10 Seconds - Test: UDP - 100Mbit Objective - Parallel: 1:
100
100
100
Average: 100 Mbits/sec
Deviation: 0.00%
iPerf 3.7:
pts/iperf-1.1.1 [Server Address: 192.168.100.100 - Server Port: 12345 - Duration: 10 Seconds - Test: UDP - 1000Mbit Objective - Parallel: 1]
Test 2 of 2
Estimated Trial Run Count: 3
Estimated Time To Completion: 1 Minute [20:07 CST]
Started Run 1 @ 20:06:39
Started Run 2 @ 20:06:56
Started Run 3 @ 20:07:14
Started Run 4 @ 20:07:31 *
Started Run 5 @ 20:07:48 *
Started Run 6 @ 20:08:05 *
Started Run 7 @ 20:08:22 *
Started Run 8 @ 20:08:39 *
Started Run 9 @ 20:08:56 *
Started Run 10 @ 20:09:14 *
Started Run 11 @ 20:09:31 *
Started Run 12 @ 20:09:48 *
Started Run 13 @ 20:10:05 *
Started Run 14 @ 20:10:22 *
Started Run 15 @ 20:10:39 *
Server Address: 192.168.100.100 - Server Port: 12345 - Duration: 10 Seconds - Test: UDP - 1000Mbit Objective - Parallel: 1:
444
496
568
577
593
595
602
595
588
598
593
592
595
605
599
Average: 576 Mbits/sec
Deviation: 7.83%
Samples: 15
No Internet Connectivity
iPerf 3.7:
pts/iperf-1.1.1
Network Test Configuration
Server Address [Use 'localhost' if wishing to benchmark the local system/server performance.]
Enter Value: 192.168.100.100
Server Port [The default iperf3 server port is 5201.]
Enter Positive Number: 12345
1: 10 Seconds
2: 30 Seconds
3: 60 Seconds
4: 360 Seconds
5: Test All Options
** Multiple items can be selected, delimit by a comma. **
Duration: 1
1: TCP
2: UDP
3: UDP - 100Mbit Objective
4: UDP - 1000Mbit Objective
5: Test All Options
** Multiple items can be selected, delimit by a comma. **
Test: 5
1: 1
2: 5
3: 10
4: 20
5: 32
6: 64
7: Test All Options
** Multiple items can be selected, delimit by a comma. **
Parallel: 1
Phoronix Test Suite v10.8.4
System Information
PROCESSOR: ARMv8 rev 0 @ 1.91GHz
Core Count: 4
Cache Size: 4 MB
Scaling Driver: tegra194 schedutil
GRAPHICS: NVIDIA TEGRA
Vulkan: 1.3.212
Screen: 1920x1080
MOTHERBOARD: EDK II 1.0-d7fb19b
BIOS Version: 1.0-d7fb19b
Network: Realtek RTL8822CE 802.11ac PCIe
MEMORY: 16GB
DISK: 16GB DG4016
File-System: ext4
Mount Options: relatime rw
Disk Details: Block Size: 4096
OPERATING SYSTEM: Ubuntu 20.04
Kernel: 5.10.104-tegra (aarch64)
Desktop: GNOME Shell 3.36.9
Display Server: X Server 1.20.13
Compiler: GCC 9.4.0
Security: itlb_multihit: Not affected
+ l1tf: Not affected
+ mds: Not affected
+ meltdown: Not affected
+ spec_store_bypass: Not affected
+ spectre_v1: Mitigation of __user pointer sanitization
+ spectre_v2: Mitigation of Branch predictor hardening
+ srbds: Not affected
+ tsx_async_abort: Not affected
Would you like to save these test results (Y/n): n
iPerf 3.7:
pts/iperf-1.1.1 [Server Address: 192.168.100.100 - Server Port: 12345 - Duration: 10 Seconds - Test: TCP - Parallel: 1]
Test 1 of 4
Estimated Trial Run Count: 3
Estimated Test Run-Time: 1 Minute
Estimated Time To Completion: 3 Minutes [21:02 CST]
Started Run 1 @ 20:59:19
Started Run 2 @ 20:59:36
Started Run 3 @ 20:59:53
Server Address: 192.168.100.100 - Server Port: 12345 - Duration: 10 Seconds - Test: TCP - Parallel: 1:
934
912
889
Average: 912 Mbits/sec
Deviation: 2.47%
iPerf 3.7:
pts/iperf-1.1.1 [Server Address: 192.168.100.100 - Server Port: 12345 - Duration: 10 Seconds - Test: UDP - Parallel: 1]
Test 2 of 4
Estimated Trial Run Count: 3
Estimated Test Run-Time: 1 Minute
Estimated Time To Completion: 3 Minutes [21:02 CST]
Started Run 1 @ 21:00:17
Started Run 2 @ 21:00:34
Started Run 3 @ 21:00:51
Server Address: 192.168.100.100 - Server Port: 12345 - Duration: 10 Seconds - Test: UDP - Parallel: 1:
1.05
1.05
1.05
Average: 1.05 Mbits/sec
Deviation: 0.00%
iPerf 3.7:
pts/iperf-1.1.1 [Server Address: 192.168.100.100 - Server Port: 12345 - Duration: 10 Seconds - Test: UDP - 100Mbit Objective - Parallel: 1]
Test 3 of 4
Estimated Trial Run Count: 3
Estimated Test Run-Time: 1 Minute
Estimated Time To Completion: 2 Minutes [21:02 CST]
Started Run 1 @ 21:01:14
Started Run 2 @ 21:01:31
Started Run 3 @ 21:01:48
Server Address: 192.168.100.100 - Server Port: 12345 - Duration: 10 Seconds - Test: UDP - 100Mbit Objective - Parallel: 1:
100
100
100
Average: 100 Mbits/sec
Deviation: 0.00%
iPerf 3.7:
pts/iperf-1.1.1 [Server Address: 192.168.100.100 - Server Port: 12345 - Duration: 10 Seconds - Test: UDP - 1000Mbit Objective - Parallel: 1]
Test 4 of 4
Estimated Trial Run Count: 3
Estimated Time To Completion: 1 Minute [21:02 CST]
Started Run 1 @ 21:02:11
Started Run 2 @ 21:02:29
Started Run 3 @ 21:02:46
Server Address: 192.168.100.100 - Server Port: 12345 - Duration: 10 Seconds - Test: UDP - 1000Mbit Objective - Parallel: 1:
600
601
601
Average: 601 Mbits/sec
Deviation: 0.10%
nvidia@ubuntu:~$ sudo jetson_clocks
nvidia@ubuntu:~$ phoronix-test-suite run pts/iperf
No Internet Connectivity
iPerf 3.7:
pts/iperf-1.1.1
Network Test Configuration
Server Address [Use 'localhost' if wishing to benchmark the local system/server performance.]
Enter Value: 192.168.100.100
Server Port [The default iperf3 server port is 5201.]
Enter Positive Number: 12345
1: 10 Seconds
2: 30 Seconds
3: 60 Seconds
4: 360 Seconds
5: Test All Options
** Multiple items can be selected, delimit by a comma. **
Duration: 1
1: TCP
2: UDP
3: UDP - 100Mbit Objective
4: UDP - 1000Mbit Objective
5: Test All Options
** Multiple items can be selected, delimit by a comma. **
Test: 5
1: 1
2: 5
3: 10
4: 20
5: 32
6: 64
7: Test All Options
** Multiple items can be selected, delimit by a comma. **
Parallel: 1
Phoronix Test Suite v10.8.4
System Information
PROCESSOR: ARMv8 rev 0 @ 1.91GHz
Core Count: 4
Cache Size: 4 MB
Scaling Driver: tegra194 schedutil
GRAPHICS: NVIDIA TEGRA
Vulkan: 1.3.212
Screen: 1920x1080
MOTHERBOARD: EDK II 1.0-d7fb19b
BIOS Version: 1.0-d7fb19b
Network: Realtek RTL8822CE 802.11ac PCIe
MEMORY: 16GB
DISK: 16GB DG4016
File-System: ext4
Mount Options: relatime rw
Disk Details: Block Size: 4096
OPERATING SYSTEM: Ubuntu 20.04
Kernel: 5.10.104-tegra (aarch64)
Desktop: GNOME Shell 3.36.9
Display Server: X Server 1.20.13
Compiler: GCC 9.4.0
Security: itlb_multihit: Not affected
+ l1tf: Not affected
+ mds: Not affected
+ meltdown: Not affected
+ spec_store_bypass: Not affected
+ spectre_v1: Mitigation of __user pointer sanitization
+ spectre_v2: Mitigation of Branch predictor hardening
+ srbds: Not affected
+ tsx_async_abort: Not affected
Would you like to save these test results (Y/n): n
iPerf 3.7:
pts/iperf-1.1.1 [Server Address: 192.168.100.100 - Server Port: 12345 - Duration: 10 Seconds - Test: TCP - Parallel: 1]
Test 1 of 4
Estimated Trial Run Count: 3
Estimated Test Run-Time: 1 Minute
Estimated Time To Completion: 3 Minutes [23:40 CST]
Started Run 1 @ 23:38:03
Started Run 2 @ 23:38:20
Started Run 3 @ 23:38:37
Started Run 4 @ 23:38:54 *
Started Run 5 @ 23:39:12 *
Server Address: 192.168.100.100 - Server Port: 12345 - Duration: 10 Seconds - Test: TCP - Parallel: 1:
918
894
872
930
907
Average: 904 Mbits/sec
Deviation: 2.48%
Samples: 5
iPerf 3.7:
pts/iperf-1.1.1 [Server Address: 192.168.100.100 - Server Port: 12345 - Duration: 10 Seconds - Test: UDP - Parallel: 1]
Test 2 of 4
Estimated Trial Run Count: 3
Estimated Test Run-Time: 1 Minute
Estimated Time To Completion: 3 Minutes [23:41 CST]
Started Run 1 @ 23:39:35
Started Run 2 @ 23:39:52
Started Run 3 @ 23:40:09
Server Address: 192.168.100.100 - Server Port: 12345 - Duration: 10 Seconds - Test: UDP - Parallel: 1:
1.05
1.05
1.05
Average: 1.05 Mbits/sec
Deviation: 0.00%
iPerf 3.7:
pts/iperf-1.1.1 [Server Address: 192.168.100.100 - Server Port: 12345 - Duration: 10 Seconds - Test: UDP - 100Mbit Objective - Parallel: 1]
Test 3 of 4
Estimated Trial Run Count: 3
Estimated Test Run-Time: 1 Minute
Estimated Time To Completion: 2 Minutes [23:41 CST]
Started Run 1 @ 23:40:32
Started Run 2 @ 23:40:49
Started Run 3 @ 23:41:06
Server Address: 192.168.100.100 - Server Port: 12345 - Duration: 10 Seconds - Test: UDP - 100Mbit Objective - Parallel: 1:
100
100
100
Average: 100 Mbits/sec
Deviation: 0.00%
iPerf 3.7:
pts/iperf-1.1.1 [Server Address: 192.168.100.100 - Server Port: 12345 - Duration: 10 Seconds - Test: UDP - 1000Mbit Objective - Parallel: 1]
Test 4 of 4
Estimated Trial Run Count: 3
Estimated Time To Completion: 1 Minute [23:42 CST]
Started Run 1 @ 23:41:30
Started Run 2 @ 23:41:47
Started Run 3 @ 23:42:04
Server Address: 192.168.100.100 - Server Port: 12345 - Duration: 10 Seconds - Test: UDP - 1000Mbit Objective - Parallel: 1:
594
600
608
Average: 601 Mbits/sec
Deviation: 1.17%
Yes. But why udp speed is slower than tcp and only happen on Jetpack 5.0.2? Generally, udp is faster than tcp beacuse it doesn’t care about packets loss and tcp has handshake mechanism to avoid packet loss then the speed will slower than udp.
we use iperf3 with the default setting for our tests usually. The result is a good estimate about how fast you can transfer images. A bit fluctuation is quite normal, but half a gigabit is something I’ve last seen on am IMX6 powered machine.
Server and client connected via LAN Cable.
Server:
iperf3 -s
Client:
iperf3 -c {server_ip}
This test yields results around 980Mbit/s with all devices whereas the Xavier NX results in ~480MBit/s
Is there any special command you’d like me to run?
Why is udp slower than tcp on Jetpack 5.0.2? If tcp can reach 1000 Mbits/sec with my test environment, I think there is no problem in my test environment. Is the situation normal?