Does GPUDirect Storage require Mellanox NIC?

[k.bodi2]

No, MLNX_OFED is a software for kernel bypass applications. You need this software “MLNX_OFED” in order to utilize DMA between PCIe devices, e.g. storage ↔ GPU. An Infiniband NIC like ConnectX-4 is not required as long as you are only working locally. See my current thread for performance aspects without any installed NIC:

GDS performance not as expected

I have reinstalled everything and switched the OS from nvme0n1 to nvme1n1 just to make sure whether there is any bottleneck because of the PCI topology. Unfortunately, it did not help. I was running the following commands in 3 separate terminals simultaneously: sudo ./gdsio -D /mnt/nvme0n1/gds_dir -d 0 -w 8 -s 1G -i 1M -x 0 -I 0 -T 300 iostat -cxzk 1 nvidia-smi dmon -i 0 -s putcm The result of the both monitoring commands is attached as a PNG. The monitoring results show that %iowait of the …