I am using the gigabit ethernet port on the orin to connect to a 5 port switch on my carrier board. The switch has a tail tag driver that needs to increase the MTU by a few bytes to add the port routing information.
For some reason, anytime the MTU is changed, a fixed offset is taken out of it and there is a line in dmesg that has to do with macsec, which shouldn’t even be enabled. I can set the MTU to anything I want, including over 1500 bytes. Any time the MTU is changed manually there is another macsec message printed.
Here is the relevant section from dmesg:
[ 9.525097] nvethernet 2310000.ethernet: Adding to iommu group 42
[ 9.530109] nvethernet 2310000.ethernet: failed to read skip mac reset flag, default 0
[ 9.538063] nvethernet 2310000.ethernet: failed to read MDIO address
[ 9.544503] nvethernet 2310000.ethernet: Failed to read nvida,pause_frames, so setting to default support as disable
[ 9.566187] nvethernet 2310000.ethernet: setting to default DMA bit mask
[ 9.584656] nvethernet 2310000.ethernet: Ethernet MAC address: 48:b0:2d:69:4b:a8
[ 9.585314] nvethernet 2310000.ethernet: Macsec not enabled
[ 9.586044] nvethernet 2310000.ethernet: Macsec: Reduced MTU: 1466 Max: 9000
[ 9.594012] nvethernet 2310000.ethernet: eth1 (HW ver: 53) created with 8 DMA channels
[ 13.442270] libphy: nvethernet_mdio_bus: probed
[ 13.561353] nvethernet 2310000.ethernet eth1: Link is Up - 1Gbps/Full - flow control off
The switch requires the host port be set up as a fixed link so here is the ethernet section in the DT:
eth0: ethernet@2310000 {
status = "okay";
nvidia,mac-addr-idx = <0>;
nvidia,max-platform-mtu = <9000>;
phy-mode = "rgmii";
fixed-link {
speed = <1000>;
full-duplex;
};
mdio {
compatible = "nvidia,eqos-mdio";
#address-cells = <1>;
#size-cells = <0>;
};
};
Why is the ethernet driver changing my MTU?
I can manually set it to 1536 and it then sets it to 1502 and works, but it seems strange that this needs to be done at all.
The second problem is that the network driver’s MTU cannot be changed while it is online and prints this error in dmesg:
[ 150.216452] nvethernet 2310000.ethernet eth1: must be stopped to change its MTU
Setting the MTU in networkManager fails since it will repeatedly try to change it while its online.
This is also a problem when the switch driver loads because it will try to change the MTU to 1502 for the tail tag, but the driver wont initialize the switch if the host network device is down.
The only workaround I can think of for this would be to write a script that takes the interface down, changes the MTU to 1536, brings it back up and then modprobes the switch driver, but all of this seems like it shouldn’t be necessary.