Issues assign representor to doca_switch

Hi,

I’m trying to run doca_switch to test my PFs and representors, but I get an error when passing the -r argument with the PCI and representor name. For example, I run:

./doca_switch – -r pci/03:00.1,pf1hpf
and it returns:
Failed to extract the device representor PCI identifier: 03:00.1,pf1hpf
Failed to parse device representor identifier “pci/03:00.1,mlx5_1”: Invalid input
Failed to parse application input: Invalid input

root@localhost:/tmp/build/switch# ./doca_switch -- -r pci/03:00.1
[03:13:44:697213][2361212960][DOCA][INF][doca_log.cpp:628] DOCA version 3.1.0105
EAL: Detected CPU lcores: 8
EAL: Detected NUMA nodes: 1
EAL: Detected shared linkage of DPDK
EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
EAL: Selected IOVA mode 'PA'
TELEMETRY: No legacy callbacks, legacy socket not created
[03:13:44:735082][2361212960][DOCA][WRN][linux_device_adapter.cpp:1622] Host PF is disabled
[03:13:44:735119][2361212960][DOCA][WRN][device_utils.cpp:643] Matching device representor not found
[03:13:44:738376][2361212960][DOCA][ERR][doca_argp.cpp:551] Failed to find device representor identifier "pci/03:00.1"


Usage: doca_switch [DPDK Flags] -- [DOCA Flags] [Program Flags]

DOCA Flags:
  -h, --help                        Print a help synopsis
  -v, --version                     Print program version information
  -l, --log-level                   Set the (numeric) log level for the program <10=DISABLE, 20=CRITICAL, 30=ERROR, 40=WARNING, 50=INFO, 60=DEBUG, 70=TRACE>
  --sdk-log-level                   Set the SDK (numeric) log level for the program <10=DISABLE, 20=CRITICAL, 30=ERROR, 40=WARNING, 50=INFO, 60=DEBUG, 70=TRACE>

Program Flags:
  -a, --device                      device
  -r, --rep                         device representor
  -exp, --expert-mode               set expert mode


[03:13:44:738487][2361212960][DOCA][ERR][switch.c:65][main] Failed to parse application input: Requested Resource Not Found

I have checked with doca_caps --list-devs and the PF and its representors are listed correctly, and I also enabled esw_multiport at runtime.

I’m not sure if the problem is the way I’m specifying the identifier or if there is an extra parameter required. Does anyone know the correct way to pass a device representor to doca_switch in this DOCA version (3.1.0105)?

This is my topology:

Thanks in advance!

LOG

root@localhost:/tmp/build/switch# devlink dev show
pci/0000:03:00.0
pci/0000:03:00.1
root@localhost:/tmp/build/switch# devlink port show
pci/0000:03:00.0/262143: type eth netdev p0 flavour physical port 0 splittable false
  function:
    hw_addr 00:00:00:00:00:00
pci/0000:03:00.0/196608: type eth netdev pf0hpf flavour pcipf controller 1 pfnum 0 external true splittable false
  function:
    hw_addr 08:c0:eb:b2:43:8a
pci/0000:03:00.0/229440: type eth netdev en3f0pf0sf0 flavour pcisf controller 0 pfnum 0 sfnum 0 splittable false
  function:
    hw_addr 02:e4:f6:75:d8:a8 state active opstate attached
pci/0000:03:00.1/327679: type eth netdev p1 flavour physical port 1 splittable false
  function:
    hw_addr 00:00:00:00:00:00
pci/0000:03:00.1/262144: type eth netdev pf1hpf flavour pcipf controller 1 pfnum 1 external true splittable false
  function:
    hw_addr 08:c0:eb:b2:43:8b
auxiliary/mlx5_core.sf.2/8781824: type eth netdev enp3s0f0s0 flavour virtual splittable false
root@localhost:/tmp/build/switch# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: oob_net0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether 08:c0:eb:b2:43:96 brd ff:ff:ff:ff:ff:ff
    altname enamlnxbf17i0
    inet 172.28.20.102/24 brd 172.28.20.255 scope global dynamic noprefixroute oob_net0
       valid_lft 2613sec preferred_lft 2613sec
    inet6 fe80::ac0:ebff:feb2:4396/64 scope link 
       valid_lft forever preferred_lft forever
3: tmfifo_net0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether 00:1a:ca:ff:ff:01 brd ff:ff:ff:ff:ff:ff
    inet 192.168.100.2/30 brd 192.168.100.3 scope global noprefixroute tmfifo_net0
       valid_lft forever preferred_lft forever
    inet6 fe80::21a:caff:feff:ff01/64 scope link 
       valid_lft forever preferred_lft forever
4: p0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1550 qdisc mq state UP group default qlen 1000
    link/ether 08:c0:eb:b2:43:8e brd ff:ff:ff:ff:ff:ff
    altname enp3s0f0np0
    inet6 fe80::ac0:ebff:feb2:438e/64 scope link 
       valid_lft forever preferred_lft forever
5: p1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1550 qdisc mq state UP group default qlen 1000
    link/ether 08:c0:eb:b2:43:8f brd ff:ff:ff:ff:ff:ff
    altname enp3s0f1np1
    inet6 fe80::ac0:ebff:feb2:438f/64 scope link 
       valid_lft forever preferred_lft forever
7: pf0hpf: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 128
    link/ether fe:12:c9:9a:a1:1d brd ff:ff:ff:ff:ff:ff
    altname enp3s0f0nc1pf0
    inet6 fe80::fc12:c9ff:fe9a:a11d/64 scope link 
       valid_lft forever preferred_lft forever
9: pf1hpf: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 128
    link/ether da:59:08:97:18:e7 brd ff:ff:ff:ff:ff:ff
    altname enp3s0f1nc1pf1
    inet6 fe80::d859:8ff:fe97:18e7/64 scope link 
       valid_lft forever preferred_lft forever
10: en3f0pf0sf0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 128
    link/ether 02:25:de:62:f6:85 brd ff:ff:ff:ff:ff:ff
    altname enp3s0f0npf0sf0
11: enp3s0f0s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
    link/ether 02:e4:f6:75:d8:a8 brd ff:ff:ff:ff:ff:ff
    inet6 fe80::e4:f6ff:fe75:d8a8/64 scope link 
       valid_lft forever preferred_lft forever
12: ovs-netdev: <BROADCAST,MULTICAST,PROMISC> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether f2:c2:22:b8:c4:b7 brd ff:ff:ff:ff:ff:ff
14: br-inside: <BROADCAST,MULTICAST,PROMISC> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether 08:c0:eb:b2:43:8f brd ff:ff:ff:ff:ff:ff
15: docker0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default 
    link/ether 02:42:c1:a9:60:4e brd ff:ff:ff:ff:ff:ff
    inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0
       valid_lft forever preferred_lft forever
    inet6 fe80::42:c1ff:fea9:604e/64 scope link 
       valid_lft forever preferred_lft forever
28: br-outside: <BROADCAST,MULTICAST,PROMISC> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether 08:c0:eb:b2:43:8e brd ff:ff:ff:ff:ff:ff
root@localhost:/home/ubuntu# /opt/mellanox/doca/tools/doca_caps --list-rep-devs
PCI: 0000:03:00.0
PCI: 0000:03:00.0
root@localhost:/tmp/build/switch# /opt/mellanox/doca/tools/doca_caps --list-devs
PCI: 0000:03:00.0
        ibdev_name                                    mlx5_2
        iface_name                                    enp3s0f0s0
        iface_index                                   12
        pci_func_type                                 SF
        sf_index                                      2
        uplink_ib_port                                1
        mac_addr                                      02:e4:f6:75:d8:a8
        ipv4_addr                                     0.0.0.0
        ipv6_addr                                     fe80:0000:0000:0000:00e4:f6ff:fe75:d8a8
        gid_table_size                                255
        GID[0]                                        fe80:0000:0000:0000:00e4:f6ff:fe75:d8a8
PCI: 0000:03:00.0
        ibdev_name                                    mlx5_bond_0
        iface_name                                    p0
        iface_index                                   4
        pci_func_type                                 PF
        uplink_ib_port                                71
        mac_addr                                      08:c0:eb:b2:43:8e
        ipv4_addr                                     0.0.0.0
        ipv6_addr                                     fe80:0000:0000:0000:0ac0:ebff:feb2:438e
        gid_table_size                                0
root@localhost:/home/ubuntu# /opt/mellanox/dpdk/bin/dpdk-devbind.py  --status

Network devices using kernel driver
===================================
0000:03:00.0 'MT42822 BlueField-2 integrated ConnectX-6 Dx network controller a2d6' if=pf0hpf,p0,en3f0pf0sf0 drv=mlx5_core unused= 
0000:03:00.1 'MT42822 BlueField-2 integrated ConnectX-6 Dx network controller a2d6' if=pf1hpf,p1 drv=mlx5_core unused= 

Other Network devices
=====================
0000:03:00.2 'BlueField DPU Family Auxiliary Communication Channel [BlueField Family] c2d1' unused=

No 'Baseband' devices detected
==============================

No 'Crypto' devices detected
============================

No 'DMA' devices detected
=========================

No 'Eventdev' devices detected
==============================

No 'Mempool' devices detected
=============================

No 'Compress' devices detected
==============================

No 'Misc (rawdev)' devices detected
===================================

No 'Regex' devices detected
===========================
root@localhost:/home/ubuntu# sudo devlink dev param show  
pci/0000:03:00.0:
  name enable_remote_dev_reset type generic
    values:
      cmode runtime value true
  name esw_port_metadata type driver-specific
    values:
      cmode runtime value true
  name flow_steering_mode type driver-specific
    values:
      cmode runtime value smfs
  name enable_roce type generic
    values:
      cmode driverinit value true
  name ct_max_offloaded_conns type driver-specific
    values:
      cmode runtime value 1000000
  name fdb_large_groups type driver-specific
    values:
      cmode driverinit value 15
  name io_eq_size type generic
    values:
      cmode driverinit value 1024
  name event_eq_size type generic
    values:
      cmode driverinit value 4096
  name esw_multiport type driver-specific
    values:
      cmode runtime value false
  name enable_eth type generic
    values:
      cmode driverinit value true
  name hairpin_num_queues type driver-specific
    values:
      cmode driverinit value 1
  name hairpin_queue_size type driver-specific
    values:
      cmode driverinit value 1024
  name enable_rdma type generic
    values:
      cmode driverinit value true
  name max_macs type generic
    values:
      cmode driverinit value 128
pci/0000:03:00.1:
  name enable_remote_dev_reset type generic
    values:
      cmode runtime value true
  name esw_port_metadata type driver-specific
    values:
      cmode runtime value true
  name flow_steering_mode type driver-specific
    values:
      cmode runtime value smfs
  name enable_roce type generic
    values:
      cmode driverinit value true
  name ct_max_offloaded_conns type driver-specific
    values:
      cmode runtime value 1000000
  name fdb_large_groups type driver-specific
    values:
      cmode driverinit value 15
  name io_eq_size type generic
    values:
      cmode driverinit value 1024
  name event_eq_size type generic
    values:
      cmode driverinit value 4096
  name esw_multiport type driver-specific
    values:
      cmode runtime value false
  name enable_eth type generic
    values:
      cmode driverinit value true
  name hairpin_num_queues type driver-specific
    values:
      cmode driverinit value 1
  name hairpin_queue_size type driver-specific
    values:
      cmode driverinit value 1024
  name enable_rdma type generic
    values:
      cmode driverinit value true
  name max_macs type generic
    values:
      cmode driverinit value 128
auxiliary/mlx5_core.sf.2:
  name flow_steering_mode type driver-specific
    values:
      cmode runtime value hmfs
  name enable_roce type generic
    values:
      cmode driverinit value true
  name ct_max_offloaded_conns type driver-specific
    values:
      cmode runtime value 4294967295
  name fdb_large_groups type driver-specific
    values:
      cmode driverinit value 15
  name io_eq_size type generic
    values:
      cmode driverinit value 1024
  name event_eq_size type generic
    values:
      cmode driverinit value 4096
  name esw_multiport type driver-specific
    values:
      cmode runtime value false
  name enable_eth type generic
    values:
      cmode driverinit value true
  name hairpin_num_queues type driver-specific
    values:
      cmode driverinit value 1
  name hairpin_queue_size type driver-specific
    values:
      cmode driverinit value 1024
  name enable_rdma type generic
    values:
      cmode driverinit value true
  name max_macs type generic
    values:
      cmode driverinit value 128

The supported ways to pass a representor to doca_switch (DOCA 3.1.0105) are those documented under “Syntax – Device and Device Representor” in the DOCA Arg Parser guide:

Current strings fail because pf1hpf is not a valid (it must be pf#…vf#…sf#…, not an ifname).

Our official guidance is:

+ Use ifname/<netdev> identifiers for both device and representors with -a and -r.

+ Validate availability with doca_caps --list-devs and --list-rep-devs.

+ If the same identifiers still can’t be resolved by DOCA, we’ll debug the DOCA device / representor discovery side; functional changes inside the doca_switch sample itself remain out of NVEX production scope. So please open a case.

Please bear in mind that doca_switch is a reference application only, not production-supported.