I tried to test DOCA file compression application, by DPU ARM as server and host as client. However, it cannot find proper representor like below
- DPU
$ lspci
00:00.0 PCI bridge: Mellanox Technologies MT42822 BlueField-2 SoC Crypto enabled (rev 01)
01:00.0 PCI bridge: Mellanox Technologies MT42822 Family [BlueField-2 SoC PCIe Bridge] (rev 01)
02:00.0 PCI bridge: Mellanox Technologies MT42822 Family [BlueField-2 SoC PCIe Bridge] (rev 01)
03:00.0 Ethernet controller: Mellanox Technologies MT42822 BlueField-2 integrated ConnectX-6 Dx network controller (rev 01)
03:00.1 Ethernet controller: Mellanox Technologies MT42822 BlueField-2 integrated ConnectX-6 Dx network controller (rev 01)
$ sudo lshw -c network -businfo
Bus info Device Class Description
========================================================
pci@0000:03:00.0 p0 network MT42822 BlueField-2 integrated ConnectX-6 Dx network controller
pci@0000:03:00.1 p1 network MT42822 BlueField-2 integrated ConnectX-6 Dx network controller
virtio@1 tmfifo_net0 network Ethernet interface
oob_net0 network Ethernet interface
pci@0000:03:00.0 pf0hpf network Ethernet interface
pci@0000:03:00.1 pf1hpf network Ethernet interface
pci@0000:03:00.0 en3f0pf0sf0 network Ethernet interface
pci@0000:03:00.0 enp3s0f0s0 network Ethernet interface
pci@0000:03:00.1 en3f1pf1sf0 network Ethernet interface
pci@0000:03:00.1 enp3s0f1s0 network Ethernet interface
$ /tmp/build/file_compression/doca_file_compression -p 03:00.0 -r 40:00.0 -f result.gz
[02:45:17:995504][1506792][DOCA][WRN][common.c:297][open_doca_device_rep_with_pci] Matching device not found
[02:45:17:995541][1506792][DOCA][ERR][comch_utils.c:390][comch_utils_fast_path_init] Failed to open Comm Channel DOCA device representor based on PCI address: Requested Resource Not Found
[02:45:17:995586][1506792][DOCA][ERR][file_compression.c:111][main] Failed to initialize a comch: Requested Resource Not Found
$ /tmp/build/file_compression/doca_file_compression -p 03:00.0 -r 03:00.0 -f result.gz
[01:37:28:926366][1480122][DOCA][INF][comch_utils.c:464][comch_utils_fast_path_init] Server waiting on a client to connect
- Host
$ lspci | grep Mell
3f:00.0 Ethernet controller: Mellanox Technologies MT28908 Family [ConnectX-6]
3f:00.1 Ethernet controller: Mellanox Technologies MT28908 Family [ConnectX-6]
40:00.0 Ethernet controller: Mellanox Technologies MT42822 BlueField-2 integrated ConnectX-6 Dx network controller (rev 01)
40:00.1 Ethernet controller: Mellanox Technologies MT42822 BlueField-2 integrated ConnectX-6 Dx network controller (rev 01)
40:00.2 DMA controller: Mellanox Technologies MT42822 BlueField-2 SoC Management Interface (rev 01)
$ sudo lshw -c network -businfo
Bus info Device Class Description
==========================================================
pci@0000:03:00.0 enp3s0f0 network Ethernet Controller X710 for 10GBASE-T
pci@0000:03:00.1 enp3s0f1 network Ethernet Controller X710 for 10GBASE-T
pci@0000:3f:00.0 enp63s0f0np0 network MT28908 Family [ConnectX-6]
pci@0000:3f:00.1 enp63s0f1np1 network MT28908 Family [ConnectX-6]
pci@0000:40:00.0 enp64s0f0np0 network MT42822 BlueField-2 integrated ConnectX-6 Dx network controller
pci@0000:40:00.1 enp64s0f1np1 network MT42822 BlueField-2 integrated ConnectX-6 Dx network controller
$ /tmp/build/file_compression/doca_file_compression -p 40:00.0 -f benchmark/datasets/fpsingle/msg_sp.sp
[10:38:20:676733][139058][DOCA][WRN][doca_compress.cpp:191][doca_compress_cap_task_compress_deflate_is_supported] devinfo 0x556557a67530: Failed to check if compress_deflate_task is supported: compress_deflate is not supported by the device
[10:38:20:680540][139058][DOCA][WRN][doca_compress.cpp:191][doca_compress_cap_task_compress_deflate_is_supported] devinfo 0x556557a6c820: Failed to check if compress_deflate_task is supported: compress_deflate is not supported by the device
[10:38:20:994800][139058][DOCA][ERR][linux_devx_obj.cpp:73][priv_doca_devx_object] Failed to create devx object with syndrome=0xe5300
[10:38:20:994863][139058][DOCA][ERR][doca_dev.cpp:2184][priv_doca_dev_devx_object_create] Failed to create devx object: failed to allocate devx object wrapper with exception:
[10:38:20:994877][139058][DOCA][ERR][doca_dev.cpp:2184][priv_doca_dev_devx_object_create] DOCA exception [DOCA_ERROR_DRIVER] with message DEVX object failed to create
[10:38:20:994886][139058][DOCA][ERR][cc_devx_2.cpp:265][cc_devx_2_register_channel_connection] Failed to create channel connection object with error DOCA_ERROR_DRIVER
[10:38:20:994890][139058][DOCA][ERR][qp_channel_2.cpp:993][cc_2_qp_register_client] client registration failed for send side
[10:38:20:994903][139058][DOCA][ERR][doca_comm_channel_2.cpp:847][doca_comm_channel_2_ep_client_connect] client registration failed for doca_comm_channel_2_ep_client_connect()
[10:38:20:994907][139058][DOCA][ERR][doca_comch_pe.cpp:393][priv_doca_comch_pe_start_cb] failed to connect on client with error = DOCA_ERROR_CONNECTION_ABORTED
[10:38:21:000534][139058][DOCA][ERR][doca_pe.cpp:1092][start_context] Progress engine 0x556557a6e740: Failed to start context=0x556557a70910. err=DOCA_ERROR_CONNECTION_ABORTED
[10:38:21:000542][139058][DOCA][ERR][comch_utils.c:535][comch_utils_fast_path_init] Failed to start comch client context: Connection aborted
[10:38:21:000555][139058][DOCA][ERR][file_compression.c:111][main] Failed to initialize a comch: Connection aborted
It is DPU mode and I tested on both DOCA 2.9.2 LTS and 2.10.0.
Device #1:
----------
Device Type: BlueField2
Part Number: MBF2H516A-EEEO_Ax_Bx
Description: BlueField-2 DPU 100GbE/EDR/HDR100 VPI Dual-Port QSFP56; PCIe Gen4 x16; Crypto Enabled; 16GB on-board DDR; 1GbE OOB management; FHHL
PSID: MT_0000000704
PCI Device Name: 0000:40:00.0
Base MAC: b8cef6d13e9c
Versions: Current Available
FW 24.43.2566 N/A
FW (Running) 24.44.1036 N/A
PXE 3.7.0500 N/A
UEFI 14.37.0014 N/A
UEFI Virtio blk 22.4.0014 N/A
UEFI Virtio net 21.4.0013 N/A
Status: No matching image found
- Should I make a new SF for comch?
- Should I change status from ETH to Infiniband?