About VMA support with Connectx-6 VPI HCAs

Hi,

Is there a possibility for a client-server application communicating through TCP sockets to benefit (in terms of throughput) from the VMA library on a RHEL8.3 (Ootpa) cluster interconnected through an infiniband network and Mellanox(Nvidia) ConnectX-6 VPI HCAs ?

The documentation (A VMA Basic Usage · Mellanox/libvma Wiki · GitHub) appears to be silent even on the subject of the availability of this bypass library.
VPI support is mentionned here [Introduction - MLNX_OFED v5.4-3.0.3.0 - NVIDIA Networking Docs (Introduction - MLNX_OFED v5.4-3.0.3.0 - NVIDIA Networking Docs)
but not for ConnectX-6 VPI.

Our compute nodes are powered by 2 Intel Ice Lake processors (40 cores each).

Software installed:

rpm -qa | grep -E “rdma|infiniband|uverbs|ucx|vma” | grep -v “kernel-kernel”

infiniband-diags-54mlnx1-1.54103.x86_64
ucx-devel-1.11.0-1.54103.x86_64
ucx-ib-1.11.0-1.54103.x86_64
librdmacm-54mlnx1-1.54103.x86_64
librdmacm-utils-54mlnx1-1.54103.x86_64
ucx-cma-1.11.0-1.54103.x86_64
rdma-core-devel-54mlnx1-1.54103.x86_64
ucx-rdmacm-1.11.0-1.54103.x86_64
rdma-core-54mlnx1-1.54103.x86_64
ucx-knem-1.11.0-1.54103.x86_64
ucx-1.11.0-1.54103.x86_64

(No libvma currently installed …)

Thanks for any information on the subject.

Jean-Marc

Hello Jean-Marc,

Thank you for posting your inquiry on the NVIDIA Developer Forum - Infrastructure and Networking - Section.

Yes, As per official documentation → NVIDIA Messaging Accelerator (VMA) Documentation Rev 9.4.0 - VMA v9.4.0 - NVIDIA Networking Docs

“The NVIDIA® Messaging Accelerator (VMA) library accelerates latency-sensitive and throughput-demanding TCP and UDP socket-based applications by offloading traffic from the user-space directly to the network interface card (NIC) or Host Channel Adapter (HCA), without going through the kernel and the standard IP stack (kernel-bypass).”

VMA will support both Ethernet and InfiniBand Technologies, and that includes the ConnectX-6 VPI adapter as well.

See the documentation link for all relevant information and use cases related to VMA.

Be aware, VMA specific support can only be obtained by an official VMA Support entitlement. The GitHub will give you some guidance but no support.

Thank you and regards,
~NVIDIA Networking Technical Support

Hi,

Requirements for VMA v9.4.0 says

  • MLNX_OFED: v5.5-1.0.3.2

I guess the VMA library needs to be installed along with an upgrade
of our OFED package if this is to have any chance of working ?

Thanks for your reply.

JM

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.