NVMeOF Target Offload on Software RAID

Infrastructure & Networking Adapters and Cables Ethernet Adapter Cards
1

Hello NVIDIA and Mellanox Teams,

I’m currently exploring NVMe over Fabrics (NVMeOF) and able to do NVMeOF on a Software RAID. from 4 NVMe SSDs. Then I encountered an issue while experimenting with the NVMeOF target offload feature alongside the Software RAID.

Following the guide HowTo Configure NVMe over Fabrics (NVMe-oF) Target Offload, I never successfully configured NVMeOF target offload with Software RAID md0. When using the command echo 1 > enable, I encountered the following error:

[   19.643528] nvmet: Couldn't find nvme pci device from device /dev/md0
echo: write error: Invalid argument

The dmesg log provided additional context:

[   19.643358] nvmet: adding nsid 10 to subsystem nvme-subsystem-name
[   19.643528] nvmet: Couldn't find nvme pci device from device /dev/md0

I’m seeking clarification on whether Software RAID is unsupported in NVMeOF target offload scenarios. I suspect that Software RAID devices do not register as NVMe devices at the kernel level, as evidenced by their absence in the /sys/class/nvme/... directory.

Any insights or guidance on resolving this issue would be greatly appreciated.

Thank you for your assistance.

My Setting:
ofed: MLNX_OFED_LINUX-5.9-0.5.6.0 (OFED-5.9-0.5.6)
Kernel: 4.15.0-20-generic
NIC: Infiniband controller: Mellanox Technologies MT27800 Family [ConnectX-5]
SSD: Samsung SSD 980 PRO 1TB → 4 such SSDs to form a software RAID

2

Hello user157267,

Thank you for posting your inquiry on the NVIDIA Developer Forum - Infrastructure and Networking - Section.

Unfortunately, Software RAID is not supported for NVME-OF. It needs to be a physical NVME device.

Thank you and regards,
~NVIDIA Networking Technical Support

1 Like
3

Thanks for the reply!

4

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.