Gigabit ethernet interface MTU automatically reduced

I am using the gigabit ethernet port on the orin to connect to a 5 port switch on my carrier board. The switch has a tail tag driver that needs to increase the MTU by a few bytes to add the port routing information.
For some reason, anytime the MTU is changed, a fixed offset is taken out of it and there is a line in dmesg that has to do with macsec, which shouldn’t even be enabled. I can set the MTU to anything I want, including over 1500 bytes. Any time the MTU is changed manually there is another macsec message printed.

Here is the relevant section from dmesg:

[    9.525097] nvethernet 2310000.ethernet: Adding to iommu group 42
[    9.530109] nvethernet 2310000.ethernet: failed to read skip mac reset flag, default 0
[    9.538063] nvethernet 2310000.ethernet: failed to read MDIO address
[    9.544503] nvethernet 2310000.ethernet: Failed to read nvida,pause_frames, so setting to default support as disable
[    9.566187] nvethernet 2310000.ethernet: setting to default DMA bit mask
[    9.584656] nvethernet 2310000.ethernet: Ethernet MAC address: 48:b0:2d:69:4b:a8
[    9.585314] nvethernet 2310000.ethernet: Macsec not enabled
[    9.586044] nvethernet 2310000.ethernet: Macsec: Reduced MTU: 1466 Max: 9000
[    9.594012] nvethernet 2310000.ethernet: eth1 (HW ver: 53) created with 8 DMA channels
[   13.442270] libphy: nvethernet_mdio_bus: probed
[   13.561353] nvethernet 2310000.ethernet eth1: Link is Up - 1Gbps/Full - flow control off

The switch requires the host port be set up as a fixed link so here is the ethernet section in the DT:

	eth0: ethernet@2310000 {
		status = "okay";
		nvidia,mac-addr-idx = <0>;
		nvidia,max-platform-mtu = <9000>;
		phy-mode = "rgmii";
		fixed-link {
			speed = <1000>;
			full-duplex;
		};
        
		mdio {
			compatible = "nvidia,eqos-mdio";
			#address-cells = <1>;
			#size-cells = <0>;
		};
		
	};

Why is the ethernet driver changing my MTU?
I can manually set it to 1536 and it then sets it to 1502 and works, but it seems strange that this needs to be done at all.

The second problem is that the network driver’s MTU cannot be changed while it is online and prints this error in dmesg:
[ 150.216452] nvethernet 2310000.ethernet eth1: must be stopped to change its MTU
Setting the MTU in networkManager fails since it will repeatedly try to change it while its online.
This is also a problem when the switch driver loads because it will try to change the MTU to 1502 for the tail tag, but the driver wont initialize the switch if the host network device is down.
The only workaround I can think of for this would be to write a script that takes the interface down, changes the MTU to 1536, brings it back up and then modprobes the switch driver, but all of this seems like it shouldn’t be necessary.

I have found the source of both issues, but will require a kernel patch to fix.
The problematic file in question is kernel/nvidia/drivers/net/ethernet/nvethernet/ether_linux.c starting at line 3860.

There is a check to verify that the network interface is down:

	if (netif_running(ndev)) {
		netdev_err(pdata->ndev, "must be stopped to change its MTU\n");
		return -EBUSY;
	}

Im curious if I could just comment this out, but dont know what the effects of an online MTU change would be.

Then a few lines down, it blindly forces a MTU change regardless if the register set for macsec is present or not.

#ifdef MACSEC_SUPPORT
	/* Macsec is supported, reduce MTU */
	if ((osi_core->mac == OSI_MAC_HW_EQOS && osi_core->mac_ver == OSI_EQOS_MAC_5_30) ||
	    (osi_core->mac == OSI_MAC_HW_MGBE && osi_core->mac_ver == OSI_MGBE_MAC_3_10)) {
		ndev->mtu -= MACSEC_TAG_ICV_LEN;
		netdev_info(pdata->ndev, "Macsec: Reduced MTU: %d Max: %d\n",
			    ndev->mtu, ndev->max_mtu);
	}
#endif /*  MACSEC_SUPPORT */

There is no way to enable/disable macsec at the device tree level it seems. I tried removing the register pointing to the macsec hardware but this check ignores it. There should be a device tree option like nvidia,macsec-enabled added.

In jetson Orin, you need to disable the inteface (ifconfig down xxxx) before modifying the MTU size.

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.