Can't seem to set LRO to "on" using ConnectX-4 Lx and Ubuntu 20.04

Sorry for the very basic question, but I am not having any success with this. I’d like to run some tests with large-receive-offload enabled but I am unable to turn the feature on. Everything I’ve read about this adapter implies that I should be able to use this feature.

$ uname -a

Linux ts-dc1-ph-j1csk93 5.4.0-84-generic #94-Ubuntu SMP Thu Aug 26 20:27:37 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

$ ethtool -i ens1f0

driver: mlx5_core

version: 4.9-3.1.5

$ sudo ethtool --show-offload ens1f0 | grep large-receive-offload

large-receive-offload: off

$ sudo ethtool --offload ens1f0 lro on

Could not change any device features

$ sudo ethtool --show-offload ens1f0 | grep large-receive-offload

large-receive-offload: off [requested on]

Any pointers would be greatly apprciated.

Forgot to mention, I can successfully modify most other offload features, but not LRO.

Software-based LRO was deprecated in kernel 4.10: ipv4: Remove inet_lro library · torvalds/linux@7bbf3ca · GitHub

Regarding Hardware LRO, Ethernet-based NICs that do not support the new API (which allows merging packets before they enter the kernel, i.e., hardware) can only use GRO.

For most intents and purposes LRO functionality has been subsumed by GRO functionality in later kernels and this was kept in mind when the drivers for these devices were written. In some cases, older kernels prior to the 4.10 kernel can have the legacy LRO functions enabled through ethtool, and in those circumstances, more information on the ethtool syntax/usage can be found here:

https://docs.mellanox.com/display/MLNXOFEDv493150/Ethtool

I suspect that this is the case for your installation based on the kernel version you mentioned. If in doubt for future implementations, you can also confirm whether the LRO value is [fixed] by checking the output of “ethtool -k ”, in which case the option can’t be toggled:

For example, results on a 3.10 kernel system:

[root@LABSYSTEM0 ~]# ethtool -k enp3s0f0 | grep receive

generic-receive-offload: on

large-receive-offload: off

receive-hashing: on

[root@LABSYSTEM0 ~]# ethtool -K enp3s0f0 lro on

[root@LABSYSTEM0 ~]# ethtool -k enp3s0f0 | grep receive

generic-receive-offload: on

large-receive-offload: on

receive-hashing: on

Results on a 5.11 kernel system:

root@LABSYSTEM2:~# ethtool -k enp5s0f0 | grep large

large-receive-offload: off [fixed]

root@LABSYSTEM2:~#

root@LABSYSTEM2:~# ethtool -K enp5s0f0 lro on

Actual changes:

rx-lro: off [requested on]

Could not change any device features

-Nvidia Network Support

Thanks very much for your detailed reply. That all makes sense. Based on everything you said I do have one followup question…

Since we are running kernel 4.9 (so, < 4.10) and since running …

ethtool --show-offload ens1f0 | grep large-receive-offload

… returns …

large-receive-offload: off

… and does NOT return …

large-receive-offload: off [fixed]

… it seems that I should be able to enable LRO, but I cannot. All attempts at running …

ethtool --show-offload ens1f0 | grep large-receive-offload

… return …

large-receive-offload: off [requested on]

It is acting as if the feature is “fixed” but it is not showing the feature as “fixed”.

Dang, cut-n-paster error in the last part of the reply above. Meant to say …

All attempts at running …

ethtool --offload ens1f0 lro on

… return …

large-receive-offload: off [requested on]

It is acting as if the feature is “fixed” but it is not showing the feature as “fixed” in the original “ethtool --show-offload ens1f0” query.