OVS Port Mirroring with CX5

Hello,

I have a Mellanox MCX512A-ACA_Ax adapter which is in switchdev mode. I have added 3 representor ports in OVS and tried running traffic between two of the ports. It works fine as expected, flows get offloaded and I get full rate.

ovs-dpctl dump-flows --names type=offloaded | grep enp

in_port(enp59s0f0_4),eth(src=fa:16:3e:95:fa:61,dst=fa:16:3e:30:0c:46),eth_type(0x0800), packets:2017511626, bytes:3206738851796, used:0.920s, actions:enp59s0f0_2

in_port(enp59s0f0_2),eth(src=fa:16:3e:30:0c:46,dst=fa:16:3e:95:fa:61),eth_type(0x0800), packets:2970848248, bytes:1509190466534, used:0.920s, actions:enp59s0f0_4

The issue is that when I try to create a mirror from one source VF to the other, the flows are not offloaded anymore and the rate drops.

ovs-vsctl – --id=@p1 get port enp59s0f0_5 – --id=@p2 get port enp59s0f0_2 – --id=@m create mirror name=m0 select-src-port=@p2 output-port=@p1 – set bridge br-int mirrors=@m

Am I doing anything wrong or is it a limitation?

My expectation was for the switch to send packets on both VFs without the packet arriving at the representor port.

Also I tried adding a flow manually to multicast the packets from vf 2 to vf 4 and 5.

ovs-ofctl add-flow br-int “table=0,priority=20,in_port=enp59s0f0_2,actions=enp59s0f0_4,enp59s0f0_5”

Again, the flow is not offloaded.

I’m running CentOS 7.5 with kernel 3.10.0-1062.12.1.el7.x86_64 and OVS 2.9.0. I also tried on CentOS 7.7 with kernel 3.10.0-1062.4.3.el7.x86_64 and OVS 2.11.0 with no luck.

Regards,

Alex

Hi Alex,

Are you working with MLNX_OFED driver?

What is the driver version?

Regards,

Chen

Hi Chen,

Found a working combination with kernel 3.10.0-1062.12.1.el7.x86_64, openvswitch-2.11.0-4.el7.src.rpm and MLNX_OFED_LINUX-5.0-1.0.0.0-rhel7.5-x86_64.

I was previously using MLNX_OFED_LINUX-4.7-3.2.9.0 which didn’t work for me.

Now port mirroring works, but I have a different issue now.

I’m trying to duplicate the packets and send them on the PF uplink with a different VLAN.

I have the following representors in my bridge:

Port “enp59s0f0_5”

tag: 6

Interface “enp59s0f0_5”

Port “enp59s0f0”

Interface “enp59s0f0”

Port “enp59s0f0_4”

tag: 4

Interface “enp59s0f0_4”

Port “enp59s0f0_2”

tag: 4

Interface “enp59s0f0_2”

Port “enp59s0f0_3”

tag: 4

Interface “enp59s0f0_3”

I then add the following flow:

ovs-ofctl add-flow br-int table=0,priority=15,in_port=enp59s0f0_2,actions=enp59s0f0_4,mod_vlan_vid:100,enp59s0f0

My goal is to duplicate packets from port enp59s0f0_2 and send them out the PF uplink (enp59s0f0) with a modified VLAN ID (100).

When I check the offloaded flows, it seems that the flow was updated successfully:

in_port(enp59s0f0_2),eth(src=fa:16:3e:30:0c:46,dst=fa:16:3e:95:fa:61),eth_type(0x0800),ipv4(frag=no), packets:99435, bytes:50382024, used:8.060s, actions:enp59s0f0_4,push_vlan(vid=100,pcp=0),enp59s0f0

Though if I capture packets on a device connected to the PF I only get a few thousand packets every 30 seconds, which is the flow aging time I set.

If I don’t use the mod_vlan_vid action all packets are duplicated just fine.

Any idea why this is happening?

Thanks,

Alex

Hi Alex,

I’m glad to hear that port mirroring is working.

Can you please try to specify the destination port: “mod_vlan_vid:100,output:<port_num>”

Please make sure that you have VLAN enabled on the physical switch that OVS is attached to & ensure the port is configured to trunk the VLAN.

Regards,

Chen