DOCA Switch application

I try to play around with the DOCA switch example following the description NVIDIA DOCA Switch Application Guide.

Following another guide, I understand now about the physical and virtual functions. So I managed to create virtual functions on the host, then I have two extra pf0vf0 and pf0vf1 devices on the DPU. The reason why I needed this because the documentation mentioned above is saying that the switch application only works with one physical port (and I wanted to switch traffic from p0 to p1).

Anyway, I started the app via this command:

# /opt/mellanox/doca/applications/switch/bin/doca_switch -a 03:00.0,representor=[0-1],dv_flow_en=2 -- -l 60
EAL: Detected CPU lcores: 8
EAL: Detected NUMA nodes: 1
EAL: Detected shared linkage of DPDK
EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
EAL: Selected IOVA mode 'PA'
EAL: Probe PCI driver: mlx5_pci (15b3:a2d6) device: 0000:03:00.0 (socket -1)
TELEMETRY: No legacy callbacks, legacy socket not created
[09:57:02:620065][921275][DOCA][DBG][switch_core.c:463][switch_ports_count] Initialize Switch with 3 ports
[09:57:02:728800][921275][DOCA][DBG][dpdk_utils.c:440][port_init] Port 0 MAC: 08 c0 eb 97 d5 52
[09:57:02:768530][921275][DOCA][DBG][dpdk_utils.c:440][port_init] Port 1 MAC: de f7 a0 0d 26 17
[09:57:02:808141][921275][DOCA][DBG][dpdk_utils.c:440][port_init] Port 2 MAC: 1e 0b 08 99 aa 03
[09:57:02:808277][921275][DOCA][WRN][engine_model.c:74][adapt_queue_depth] adapting queue depth to 128.

The documentation says to create an fwd first, so I did that:

SWITCH>> create fwd type=port,port_id=0xffff

This time, i don’t really know what this port_id is. But converting it into decimal, it is 65535, which is the representor of the physical function of the host PF0 port on the DPU, i.e., this is the pf0hpf.

Then, the next command is this:

SWITCH>> create pipe port_id=0,name=p0_to_vf1,root_enable=1,fwd=1
[10:00:35:298676][921275][DOCA][DBG][switch_core.c:85][pipe_create] Create pipe is being called
[10:00:35:326972][921275][DOCA][INF][switch_core.c:120][pipe_create] Pipe created successfully with id: 16057732822350369178

I understand the pipe thing that you create a pipe for each ingress port. So, this will create a pipe for the incoming port p0 on the DPU. I assign the name, enable root (which is probably telling the system that this is the main/first/root table packets will be matched on). The only thing I cannot get my head around is the fwd=1 thingy. Is this referring to the first fwd element I created just before? Because that command did not return me any ID, and the port_id is actually 65535 not 1. Anyway, the command is successfully executed.

Then, again, I created another fwd:

SWITCH>> create fwd type=port,port_id=1

Lastly, I add an entry to the pipe:

SWITCH>> add entry pipe_queue=0,fwd=1,pipe_id=16057732822350369178
[10:59:08:818677][921275][DOCA][DBG][switch_core.c:150][pipe_add_entry] Add entry is being called
[10:59:08:818925][921275][DOCA][INF][switch_core.c:194][pipe_add_entry] Entry created successfully with id: 7142399667034538599

The entry again is having a fwd=1, which I am not sure about.

To see how an entry actually looks like, I wanted to query it. So I did this:

query entry_id=7142399667034538599
[11:28:49:490647][921275][DOCA][DBG][switch_core.c:339][flow_query] Query is being called
[11:28:49:490781][921275][DOCA][ERR][dpdk_flow.c:827][dpdk_flow_query] Port 0 query flow fail, type 1 message: counter not defined in the rule
[11:28:49:490830][921275][DOCA][ERR][dpdk_pipe_legacy.c:1129][dpdk_pipe_entry_query] failed to query entry - query entry rc=-22
[11:28:49:490900][921275][DOCA][ERR][switch_core.c:348][flow_query] Query on entry failed
[11:28:49:490932][921275][DOCA][INF][flow_parser.c:2011][cmd_query_parsed] Total bytes: 8192
[11:28:49:490956][921275][DOCA][INF][flow_parser.c:2012][cmd_query_parsed] Total packets: 4294967294

As can be seen, it gives me an error saying counter not defined in the rule. However, I did not find any setting during entry addition that would le me define such a counter. Maybe it’s an autoincremented counter that the switch is doing automatically, but then it is not working properly?

Any idea about the errror and the fwd things?

BTW: I am running the latest Bluefield 2 BFB image with doca 2.6 on the Bluefield