Please refer to DPDK’s official programmer’s guide for programming guidance as well as relevant BlueField platform and DPDK driver information on using DPDK with your DOCA application on BlueField-2
I have had the chance to get my hands dirty with BF-2 DPUs deployed at Cloudlab. However, since everyone is now crazy about leasing resources at the Clemson site, I did not finish everything I wanted to do.
If you don’t want to jump from pointers to pointers, I am making a series of blog posts about the first impressions.
Before talking about DPDK programming resources, it would be great if we could run standard DPDK applications on the Bluefield without any error.
Just as usual, I was following an “official” blogpost about how to get OvS-DPDK working on the Bluefield, but of course it is not working out-of-the-box :) sorry, sarcasm :D
Once the ports are added to OvS-DPDK, I got the error of “address family not supported by protocol”. I remember the “good”-old days when playing around with OvS-DPDK on servers with dumber Mellanox ConnectX-5, and I had similar problem due to the mlx5 driver. After compiling everything on my own, it was working back then…should I do the same here again?
I was so happy after upgrading the OS to the freshest one and saw all things are supplied by default, like ovs, doca, dpdk…
Anyone encountered the same issue before?
The blog I mentioned above was lacking of the RDMA-related setting found in another official blog that suggested to enable RDMA after switching the mode of operation from SEPARATED_HOST mode to EMBEDDED/SMARTNIC mode.
Thanks
What software and versions are running on the Bluefield-2? The installation guide in the DOCA SDK documentation is the best resource to make sure dependencies are installed correctly on both the host and BF2 sides:
When using the DOCA BFB image (DOCA_v1.0_BlueField_OS_Ubuntu_20.04-5.3-1.0.0.0-3.6.0.11699-1-aarch64.bfb), the BF2 gets setup and preconfigured in Embedded mode with all of the representer ports created and added to ovs bridges.
Thanks, @jubetz , I used the official guide and I am running the latest firmware with DOCA and every other package (e.g., dpdk, ovs, spdk) pre-configured and installed.
Still, I have the issue.
I tried running dpdk-testpmd app, which did not find a shared library.
/opt/mellanox/dpdk/bin/dpdk-testpmd: error while loading shared libraries: librte_ethdev.so.21: cannot open shared object file: No such file or directory
So, I rather removed all OvS and DPDK packages and started to reinstall DPDK from scratch on the Bluefield.
The new dpdk-testpmd at least runs now…let’s see if I send some packets to it, it works properly.
On the other hand, I have been a heavy DPDK user for ages (mostly x86 and OvS-DPDK) but I have never seen and used this representor thing in the EAL options like this:
ovs-vsctl set Open_vSwitch . other_config:dpdk-extra=“-w <pcie_device_id>,representor=[0,65535]”
I only used the pcie_device_id to identify a port. Can you let me know what this representor means in this environment and whether it is necessary to be used?
Can you try to add this environment variable, then try to run? See if that works?
export LD_LIBRARY_PATH=/opt/mellanox/dpdk/lib/aarch64-linux-gnu/
Thanks! I tried. Now, testpmd works and can find the libraries.
I tried, however, to compile dpdk-pktgen from source.
It complains about not finding libdpdk and libmlx4, which is strange. I have installed the dev libraries for libdpdk, but of course I did not install any mlx4 driver.
Run-time dependency libdpdk found: NO (tried cmake)
meson.build:59:0: ERROR: Could not generate cargs for libdpdk:
Package libmlx4 was not found in the pkg-config search path.
Perhaps you should add the directory containing `libmlx4.pc'
to the PKG_CONFIG_PATH environment variable
Package 'libmlx4', required by 'libdpdk', not found
Have you encountered such an error so far?
thanks
3 Likes