Behavior of Mellanox NIC

Infrastructure & Networking Software And Drivers SoC And SmartNIC
1

We generated 8 virtual functions for each PF (MELLANOX NIC). We used those VFs to deploy the virtual machine. We are unable to shut off the VM ports by shutting down the physical link.

2

Hi,

Please follow the suggestions below:

  1. There is a firmware feature called keep link up, that means the physical link stays up even if you put the interface down. Please make sure the mlxconfig “KEEP_ETH_LINK_UP_P1” or “KEEP_IB_LINK_UP_P1” as expected.

     KEEP LINK UP:
             KEEP_ETH_LINK_UP_P1=<False|True>        When set, the NIC keeps the link up as long as the server is
                                                     not in standby mode (Ethernet only).
             KEEP_ETH_LINK_UP_P2=<False|True>       

             KEEP_IB_LINK_UP_P1=<False|True>         When set, the NIC keeps the link up as long as the server is
                                                     not in standby mode (InfiniBand only).
             KEEP_IB_LINK_UP_P2=<False|True>

Use the “# mlxconfig -d xx:xx.x set KEEP_ETH_LINK_UP_P1=0” to configure, and use “mlxfwreset -d xx:xx.x r -y” to reset the FW and make it work.

  1. If you want to test the PF port down and check VF state, please don’t use the ip link down or ifdown or ifconfig down on PF port. you should disable/shutdown the correpsonding port on the switch and check.

  2. Use the “ip link show” command to see if the vf port link-state is “auto”.

https://man7.org/linux/man-pages/man8/ip-link.8.html

                  state auto|enable|disable - set the virtual link
                  state as seen by the specified VF. Setting to auto
                  means a reflection of the PF link state, enable
                  lets the VF to communicate with other VFs on this
                  host even if the PF link state is down, disable
                  causes the HW to drop any packets sent by the VF.
3

Hi,

Thanks for the response. We have set vf link-state auto, but still we are observing the issue.

mlxconfig is not working on compute node. We tried to install by downloading the pkg from mellanox official site but no luck

Could you please help us to proceed further?

Thanks,
Susmitha

4

Please download and install the mft 4.23.0 version onto your system, then you can use the mlxconfig to modify the fw config.