Driver: in-tree mlx5_core (upstream kernel driver, not DOCA — DOCA is not installed on either host and isn’t required, since we have a working reference host on the same driver stack). Kernel 7.1.8, built with CONFIG_MLX5_ESWITCH=y and CONFIG_MLX5_SW_STEERING=y.
Symptom: Two hosts, identical NIC PSID/firmware/part number, same kernel/driver build:
Host A: flow_steering_mode defaults to smfs at boot. Hardware IPsec packet offload (ip xfrm … offload packet) works, validated under RoCEv2 traffic with correct ethtool -S IPsec counters.
Host B: flow_steering_mode defaults to dmfs. IPsec offload install fails (Device failed to offload this state), and manually forcing devlink dev param set … flow_steering_mode value smfs fails immediately with Software managed steering is not supported by current device.
Question: Is there a way to reset/force the card from its current dmfs state to smfs
Thank you for posting your query on our community.
Could you please compare the RoCE enablement setting on both hosts? On each host, run the following against the ConnectX‑6 Dx PCI function:
sudo devlink dev param show pci/<BDF> name enable_roce
sudo devlink dev param show pci/<BDF> name flow_steering_mode
Replace <BDF> with the relevant PCI address, such as 0000:10:00.0. For enable_roce, true indicates enabled and false indicates disabled. This parameter is documented in the upstream mlx5 driver documentation.
Please share the output from both hosts. If RoCE is disabled on the affected host, please test enabling it and retry selecting SMFS. Please note that changing enable_roce requires a driver reload to take effect.
These systems have been running RoCEv2 for some time. There is no issue with RoCE.
Its when I try to enable ipsec, I notice I cannot enable on those cards which are set to dmfs.