AGX Orin - mtu 1466

Autonomous Machines Jetson & Embedded Systems Jetson AGX Orin
1

Is this a bug or a feature?

Why is the default MTU on Orin set to 1466?
Why do I need to do some weird stuff to set it to 1500?

root@orin:~# ifconfig eth0
eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1466
        inet 192.168.5.95  netmask 255.255.255.0  broadcast 192.168.5.255
        inet6 fe80::4ab0:2dff:fe69:4126  prefixlen 64  scopeid 0x20<link>
        ether 48:b0:2d:69:41:26  txqueuelen 1000  (Ethernet)
        RX packets 76  bytes 14899 (14.8 KB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 105  bytes 22817 (22.8 KB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
root@orin:~# ifconfig eth0 down
[  253.459976] Trying to unregister non-registered hwtime source
root@orin:~# ifconfig eth0 mtu 1500
root@orin:~# ifconfig eth0 up
root@orin:~# ifconfig eth0
eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1466
        inet 192.168.5.95  netmask 255.255.255.0  broadcast 192.168.5.255
        inet6 fe80::4ab0:2dff:fe69:4126  prefixlen 64  scopeid 0x20<link>
        ether 48:b0:2d:69:41:26  txqueuelen 1000  (Ethernet)
        RX packets 76  bytes 14899 (14.8 KB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 105  bytes 22817 (22.8 KB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
root@orin:~# ifconfig eth0 down
[  463.366881] Trying to unregister non-registered hwtime source
root@orin:~# ifconfig eth0 mtu 1534
root@orin:~# ifconfig eth0 up
root@orin:~# ifconfig eth0
eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 192.168.5.95  netmask 255.255.255.0  broadcast 192.168.5.255
        inet6 fe80::4ab0:2dff:fe69:4126  prefixlen 64  scopeid 0x20<link>
        ether 48:b0:2d:69:41:26  txqueuelen 1000  (Ethernet)
        RX packets 155  bytes 30295 (30.2 KB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 190  bytes 54210 (54.2 KB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
3

I will check with internal to see if can have the update, but any issue met with 1466 or you can try with 1500 directly to see if any impact.

4

Since the device runs IP routing, having MTU’s that don’t match may result unnecessary ip fragmentation and performance penalty.

I have also verified that with MTU 1466, the device can receive standard size frames, but is not able to send them, since the IP layer is informed otherwise.


19:35:53.585976 00:04:4b:e5:9d:57 > 48:b0:2d:69:41:26, ethertype IPv4 (0x0800), length 1514: 10.0.38.101 > 10.0.38.253: ICMP echo request, id 39, seq 1, length 1480
19:35:53.586222 48:b0:2d:69:41:26 > 00:04:4b:e5:9d:57, ethertype IPv4 (0x0800), length 1474: 10.0.38.253 > 10.0.38.101: ICMP echo reply, id 39, seq 1, length 1440
19:35:53.586278 48:b0:2d:69:41:26 > 00:04:4b:e5:9d:57, ethertype IPv4 (0x0800), length 74: 10.0.38.253 > 10.0.38.101: ip-proto-1

I am currently forcing MTU to 1534. In the short term, it seems to work, but need to do more testing.
It looks to me like the nvethernet driver is confused about the definition of MTU.

6

@kayccc, have you received any more information internally? Do you have any suggested actions we should take?

7

Is this issue solved?

8

No, it has not been resolved.

9

You may use a rc.local startup script as workaround:

As root, create and edit file /etc/rc.local such as:

#!/bin/sh

ip link set down eth0
ip link set eth0 mtu 1534
ip link set up eth0

Then change permissions:

chmod 755 /etc/rc.local

and reboot.

10

Question for @WayneWWW or anyone from NVIDIA…

I was trying to find an answer for @Honey_Patouceul on setting this automatically. The value of mtu which is in question can be viewed in “/sys/class/net/eth0/mtu”. “cat” verifies values from ifconfig. The actual configuration should be via “/etc/sysctl.conf” (or one of the “/etc/sysctl.d/*” files). These lines should be reflected in:
find /proc/sys/net -name mtu

Editing the “/etc/sysctl.conf” (or the sysctl.d/ subdirectory) should work to edit those files at boot:

net.ipv6.conf.eth0.mtu = 1534
net.ipv4.conf.eth0.mtu = 1534

(or 1500, but I was trying to make sure and tried 1534 as well)

Those sysctl edits do not make the update. I’ve concluded that somehow, during the transition to the 5.x kernel (of L4T R35.2.1, which I’m testing on) the sysctl.conf mechanism was lost. Would anyone at NVIDIA know why sysctl.conf no longer works for networking MTU?

Incidentally, it should be possible to pass a driver argument (it is a Realtek driver) to the kernel command line at boot to adjust this MTU, but I was unable to find a list of valid kernel arguments for the Realtek driver, so I could not try.

1 Like
11

The workaround with rc.local may work better or worse, but is is only a workaround - not a fix. Firstly, the rc.local has been obsolete for several years and not recommended to use.
Secondly putting the script ip up/down in the script that is run when boot process is essentially completed, causes unnecessary network flapping. There are better ways for setting custom MTU and vary depending on the netwaork management software (NetworkManager or networkd).
However, the point is that if you are using the system API to set MTU is does not work correctly. So, if you have software that relies of this API, e.g. anything related to tunneling, it will be broken.
The bug is in the nvethernet driver and it still has not been fixed in Jetpack 5.1. Nvidia has not even classified it as a bug, even though I reported it 4 months ago.

13

Thanks. This is the only solution for the mgbe* cards on AGX Orin which do not allow setting MTU of a connected Ethernet link.

I even tried a /etc/network/if-pre-up.d script and /etc/NetworkManager/dispatcher.d/pre-up.d script, but it did not work (the NM script was being run too late and the network script was not run for my interface at all).