I’m running jetson TX2 as a router to other jetsons and I have serious problems with peformance of network packet forwarding.
On any direct connection on my setups I can achieve 960 mbit/s without any problems, so each single
network connection is ok. Problem begins when I start using jetson tx2 as a network switch/router.
I have two setups:
laptop - (i350 = i350 on jetson tx2) - eqos on xavier
laptop - (i350 = eqos on jetson tx2) - eqos on xavier
First setup is faster, second doesn’t work at all.
I can achive 900 mbit on first setup when I put MTU on both links to 9000.
This is not useable because laptop simulates wide outer network where nobody will give me 9000,
there will be 1500.
When MTU is lower than 9000, both setups cannot give me even 900 mbit/s of throughput with
different symptoms.
Setup 1 (two i350 NIC on TX2).
When MTU is lower than 9000, CPU0 is used on 99% and speed goes down to 600 mbit/s.
I use iperf3 to benchmark with very simple settings:
laptop:
iperf3 -s
xavier:
iperf3 -c laptop
With this setup middle jetson tx2 shows that his CPU0 is busy on 100% (tegrastats and htop) if MTU is less than 9000.
It is impossible to change interrupts, all ways via /proc and /sys says about read/write error or something like this.
It seems to be impossible to change interrupt handling core =( Maybe I’m wrong and there is some way to bind
NIC to non-first core?
When MTU is 9000, CPU0 usage floats around 60-80% and forwarding traffic is about 900-950 mbit/s
Setup 2 (i350 and builtin eqos on TX2).
Traffic is floating from 40 to 270 mbit/s and very unstable:
[ 4] 3.00-4.00 sec 44.6 MBytes 374 Mbits/sec 1 402 KBytes
[ 4] 4.00-5.00 sec 29.8 MBytes 250 Mbits/sec 0 1.41 KBytes
[ 4] 5.00-6.00 sec 1.07 MBytes 8.97 Mbits/sec 1 489 KBytes
[ 4] 6.00-7.00 sec 20.7 MBytes 173 Mbits/sec 1 197 KBytes
[ 4] 7.00-8.00 sec 54.9 MBytes 460 Mbits/sec 0 349 KBytes
[ 4] 8.00-9.00 sec 33.1 MBytes 278 Mbits/sec 0 1.41 KBytes
CPU0 on Tx2 is used on 10-60% and its usage is very unstable. So this setup is not useable at all.
Looks like eqos us not stable.
Do I have any chance to achive 1 gbit/s? I was very surprised that in era of 400 gbit/s NIC we have problems with 1 gbit/s =((