We have a special use case where we simply want to process and forward packets. Is there a way to completely disable VLAN priority? As far as we can tell it is disabled, but it could be that the card is processing the packets differently for the different levels. We believe there is a way to do this, but we cannot find it in the documentation. We are using DPDK.
Hi,
Please see the below explanation. In your case you need mlnx_qos solution, as your adapter is connectx-5, and it doesn’t allow to configure it in firmware.
- mlnx_qos command. Doesn’t survive the reboot, but can be added to systemd service to load on boot
i.e.
[root@l-csi-gso-0282s ~]# vim /usr/local/bin/mlnx_qos_persistent.sh
[root@l-csi-gso-0282s ~]#
[root@l-csi-gso-0282s ~]# chmod +x /usr/local/bin/mlnx_qos_persistent.sh
[root@l-csi-gso-0282s ~]# systemctl daemon-reload
[root@l-csi-gso-0282s ~]# cat /etc/systemd/system/mlnx_qos_persistent.service
[Unit]
Description=Configures mlnx_qos persistently after the boot
[Service]
Type=oneshot
ExecStart=/usr/local/bin/mlnx_qos_persistent.sh
StandardOutput=journal+console
StandardError=journal+console
[Install]
WantedBy=network-online.target
[root@l-csi-gso-0282s ~]# cat /usr/local/bin/mlnx_qos_persistent.sh
#!/bin/bash
mlnx_qos -i enp175s0f0 --trust=dscp
for dscp in {0…63}; do sudo mlnx_qos -i enp175s0f0 --dscp2prio set,$dscp,0; sleep .000001;done
[root@l-csi-gso-0282s ~]#
[root@l-csi-gso-0282s ~]# systemctl enable --now mlnx_qos_persistent
Created symlink /etc/systemd/system/network-online.target.wants/mlnx_qos_persistent.service → /etc/systemd/system/mlnx_qos_persistent.service.
[root@l-csi-gso-0282s ~]#
- The second option is supported with CX6 DX adapters and up.
You will need to enable DSCP in fw.
By default it’s 1, and you need to set it to 2 using mstconfig command.
[root@l-csi-1708d mstflint]# ./mlxconfig/mstconfig -d 17:00.0 -e q | grep TRUST
QOS_TRUST_STATE_P1 TRUST_PCP(1) TRUST_PCP(1) TRUST_PCP(1)
QOS_TRUST_STATE_P2 TRUST_PCP(1) TRUST_PCP(1) TRUST_PCP(1)
[root@l-csi-1708d mstflint]#
mstconfig -d <mst_dev> set NUM_OF_TC_P1=1 NUM_OF_TC_P2=1
You will need to reboot/power cycle the server to make it working
If your current mstconfig version doesn’t allow you to change it, please clone the below repo and compile it from source.
Best Regards,
Anatoly
This worked perfectly!!! Thank you so much. I really appreciate the help!
-Pete
Is there some solution (disable VLAN priority) available for CX-4 cards? @abirman
Hi,
The above should work on Cx4 adapter, as firmware configuration is supported starting CX6 DX adapter only.
Best Regards,
Anatoly
Hello,
I have CX6, the above procedure don’t work for me, can you please help me, how i can disable VLAN prioritization?
mlxconfig -d /dev/mst/mt4127_pciconf3 set NUM_OF_TC_P1=2 NUM_OF_TC_P2=2
Device #1:
----------
Device type: ConnectX6LX
Name: MCX631102AS-ADA_Ax
Description: ConnectX-6 Lx EN adapter card; 25GbE; Dual-port SFP28; PCIe 4.0 x8; Secure Boot; No Crypto;
Device: /dev/mst/mt4127_pciconf3
Configurations: Next Boot New
NUM_OF_TC_P1 _2_TCs(2) _2_TCs(2)
NUM_OF_TC_P2 _2_TCs(2) _2_TCs(2)
Apply new Configuration? (y/n) [n] : y
Applying... Done!
-I- Please reboot machine to load new configurations.```
After reboot nothing is changed:
mlxconfig -d /dev/mst/mt4127_pciconf3 query | grep QOS_TRUST_STATE
QOS_TRUST_STATE_P1 TRUST_PCP(1)
QOS_TRUST_STATE_P2 TRUST_PCP(1)
ThanksHi,
You can try either cold boot (not a reboot) of the server in order to apply the configuration, or try the below mlxfwreset command:
mlxfwreset -d -y reset
To check if your configuration is applied (note the -e flag, which will print the current, the default, and next boot config):
mlxconfig -d -e q
Best Regards,
Anatoly
1 Like
Hi Anatoly,
it works perfectly now. Thanks alot.