Certainly, here’s a refined version of your text:
Hello,
We recently purchased two Mellanox ConnectX-6 DX NICs specifically for their hardware offloading capabilities. We are utilizing NFtables with flowtable, and it’s my understanding that we can enable hardware offloading using the hw-tc-offload feature. The feature is enabled, and the kernel is configured to utilize it. However, we are encountering an issue where we receive the message “Operation not supported” from the driver.
Has anyone else experienced a similar issue and found a resolution?
When you are referring to the message “Operation not supported” from the driver", are you referring to our MLNX_OFED or MLNX_EN driver? (End-to-End Networking Solutions | NVIDIA) You can validate with modinfo mlx5_core, the mlx5_core OS Inbox driver is not from Nvidia and is directly handled by the OS vendor.
What led to the “operation not supported” notification?
Are you following any deployment guidelines from Nvidia?
Check that CONFIG_NFT_FLOW_OFFLOAD is enabled in the kernel.
Install MLNX_OFED or MLNX_EN driver.
The “Operation not Supported” came from nftables after try to load the config.
root@dus6:~# cat /boot/config-6.1.4-060104-generic | grep CONFIG_NFT_FLOW_OFFLOAD
CONFIG_NFT_FLOW_OFFLOAD=m
root@dus6:~# lsmod | grep offload
nft_flow_offload 20480 0
nf_flow_table 40960 2 nft_flow_offload,nf_flow_table_inet
nf_conntrack 180224 9 xt_conntrack,nf_nat,nft_flow_offload,nft_ct,xt_state,nft_nat,xt_nat,nf_conntrack_netlink,nf_flow_table
nf_tables 282624 127058 nft_flow_offload,nft_ct,nft_compat,nft_nat,nft_chain_nat,nf_flow_table_inet
root@dus6:~# modinfo mlx5_core
filename: /lib/modules/6.1.4-060104-generic/updates/dkms/mlx5_core.ko
alias: auxiliary:mlx5_core.eth-rep
alias: auxiliary:mlx5_core.eth
basedon: Korg 6.3-rc3
version: 23.10-1.1.9
license: Dual BSD/GPL
description: Mellanox 5th generation network adapters (ConnectX series) core driver
author: Eli Cohen <eli@mellanox.com>
srcversion: 7D45852F017C3505B1609EB
alias: pci:v000015B3d0000A2DFsv*sd*bc*sc*i*
alias: pci:v000015B3d0000A2DCsv*sd*bc*sc*i*
alias: pci:v000015B3d0000A2D6sv*sd*bc*sc*i*
alias: pci:v000015B3d0000A2D3sv*sd*bc*sc*i*
alias: pci:v000015B3d0000A2D2sv*sd*bc*sc*i*
alias: pci:v000015B3d00001023sv*sd*bc*sc*i*
alias: pci:v000015B3d00001021sv*sd*bc*sc*i*
alias: pci:v000015B3d0000101Fsv*sd*bc*sc*i*
alias: pci:v000015B3d0000101Esv*sd*bc*sc*i*
alias: pci:v000015B3d0000101Dsv*sd*bc*sc*i*
alias: pci:v000015B3d0000101Csv*sd*bc*sc*i*
alias: pci:v000015B3d0000101Bsv*sd*bc*sc*i*
alias: pci:v000015B3d0000101Asv*sd*bc*sc*i*
alias: pci:v000015B3d00001019sv*sd*bc*sc*i*
alias: pci:v000015B3d00001018sv*sd*bc*sc*i*
alias: pci:v000015B3d00001017sv*sd*bc*sc*i*
alias: pci:v000015B3d00001016sv*sd*bc*sc*i*
alias: pci:v000015B3d00001015sv*sd*bc*sc*i*
alias: pci:v000015B3d00001014sv*sd*bc*sc*i*
alias: pci:v000015B3d00001013sv*sd*bc*sc*i*
alias: auxiliary:mlx5_core.sf
depends: mlx_compat,mlxdevm,psample,tls,pci-hyperv-intf,mlxfw
retpoline: Y
name: mlx5_core
vermagic: 6.1.4-060104-generic SMP preempt mod_unload modversions
sig_id: PKCS#7
signer: dus6 Secure Boot Module Signature key
sig_key: 53:5E:59:3E:AA:B7:9D:49:54:1A:D2:22:4A:0C:B0:D7:E5:89:98:0A
sig_hashalgo: sha512
signature: 11:CD:51:AB:4F:B2:EB:04:73:EC:AE:8D:F8:CD:4E:A0:8C:4D:CF:00:
F2:0D:B8:DD:9A:3F:DE:02:1E:FF:F8:91:52:1D:EB:F9:5F:0B:7E:55:
AB:A8:93:D8:42:EF:83:78:C6:E3:6C:66:6C:EC:07:E6:80:67:AB:09:
FC:8B:60:1E:F7:1C:09:35:2E:E2:90:B8:86:B6:E6:FB:DC:52:01:31:
0E:39:73:F2:29:18:89:8C:46:08:EC:B0:15:62:23:1B:C2:4E:D8:D2:
F5:9B:09:FE:BB:61:D7:EC:B0:6F:D0:76:A8:D2:C1:DC:CA:C3:A1:99:
F2:AE:E4:61:96:66:86:05:4B:86:1C:CC:05:63:E7:19:B9:29:38:B4:
5F:34:DA:C5:38:A3:99:FC:3E:06:76:90:D8:5F:2B:0B:83:3C:55:A5:
11:3B:D8:C9:60:62:81:2D:CD:97:E1:A4:EF:A1:C5:81:4E:D1:89:7F:
00:FC:FB:58:D7:2C:53:54:93:93:C1:09:88:75:C4:87:5A:3F:D5:8B:
AB:89:76:C3:E6:02:53:38:E9:B2:F4:8A:1B:5D:1B:BA:FD:73:25:84:
E6:F5:B3:98:6C:A4:2B:3D:A5:4F:46:FF:B1:B0:2C:7C:B9:78:D0:00:
09:E2:F0:8D:AD:E5:A8:DF:82:CE:F6:9F:39:24:94:69
parm: num_of_groups:Eswitch offloads number of big groups in FDB table. Valid range 1 - 1024. Default 15 (uint)
parm: debug_mask:debug mask: 1 = dump cmd data, 2 = dump cmd exec time, 3 = both. Default=0 (uint)
parm: prof_sel:profile selector. Valid range 0 - 3 (uint)
parm: probe_vf:probe VFs or not, 0 = not probe, 1 = probe. Default = 1 (bool)
What exactly are you loading, running (elaboration)?
So to understand the correlation with our driver & hw-tc-offload (which I am assuming your enabled via ethtool)
Do you have the full error message?
I try to load a single flowtable with:
table inet x {
flowtable f {
hook ingress priority 0; devices = { ens1f0np0, ens1f1np1 };
flags offload;
}
}
And i get
root@dus6:~# nft -f /etc/nftables.conf
/etc/nftables.conf:2:19-19: Error: Could not process rule: Operation not supported
flowtable f {
^
root@dus6:~#
If i remove “flags offload” it works but only in Software.
And of course hw-tc-offload is active