Hello, I’m going to set up a server-IPU node cluster by connecting multiple mellanox X5 devices to several PCs.
The OS includes a mix of Windows 10 and Linux.
I’m using C++ as the programming language.
I’ve installed the WinOF driver, but I’m aiming to transfer RDMA data using C++ source code
(rather than SMB DIRECT)
The reason for this approach is to directly send images loaded in RAM to the RAM of other nodes. SMB appears to be SSD-to-SSD transfer.
Could you please let me know which C++ APIs I should use on both Windows and Linux operating systems for performing RDMA data transmission and reception? (Server-Windows10 OS, IPU Node-Ubuntu 20.04LTS OS)
Hello user85507, and welcome to the NVIDIA Developer Community!
With Windows, RDMA is supported via the use of the NetworkDirect SPI.
Whereas with Linux, RDMA support is made possible via the use of verbs (rdma-core).
Unfortunately, these RDMA implementations are not compatible with each other.
As a result, RDMA communication from Windows to Linux (or Linux to Windows) is not in our test or support matrix.
If you are experiencing difficulties developing a custom solution, our Sales team can help connect you to the proper resources within NVIDIA to assist with design and implementation: https://www.nvidia.com/en-us/contact/sales/
Regarding GPUDirect, GPUDirect for video is supported on Windows:
For more information about GPUDirect, please visit the landing page: