Packet drop if IP TOS contains priority bits


I have ConnectX-6 LX card, and i’m trying to push some IP traffic through it using DPDK app. In most cases, this works as expected, but, if input traffic have random IP TOS bits set, then card starts to drop traffic.

More precisely, if all incoming traffic have the same value for IP TOS, everything works as expected. But if incoming traffic have different values for IP TOS, and for each IP TOS value packet rate is >100k pps, then i’m unable to push more than ~3M pps of total traffic through single VF. Application is using SRIOV, and running inside of VM on hypervisor. Hypervisor is Openstack (KVM).

For example, if IP TOS is set to 5, we can push >3M pps on single VF. If incoming traffic have ~1M5 pps of packets with IP TOS set to 5 and ~2M pps of packets with IP TOS set to 0, then maximum traffic that is coming to DPDK app is capped to ~3M pps.

Does anyone else have this issue?

Please note that i’m experiencing the same issue on both ConnnectX-4 LX and ConnectX-5 cards. In case of ConnecX-4 LX, the following commands make this problem disappear:

mstmcra $businfo 0x24ab4.0:4 0
mstmcra $businfo 0x24ab8.0:4 0
mstmcra $businfo 0x24ab4.4:4 0
mstmcra $businfo 0x24ab8.4:4 0

The same “commands” does not work for other cards. Also, i do not have a clue what those commands do - i just found them somewhere.

If anyone know what those commands do, please can provide them for other cards also?




The mcra commands you mentioned are disabling priority check on the firmware level.
It’s not recommended to modify it unless you exactly know what it does.

The described issue requires a deeper investigation, and more detailed information is required in order to do so (i.e. OS/driver/firmware/DPDK versions)

In order to perform such investigation, we recommend to open a support case in Nvidia portal, and we will be happy to assist you.

Best Regards,