Windows 2016 Storage Spaces Direct over IPoIB

Infrastructure & Networking Software And Drivers WinOF Driver
1

Hello,

I am in need of some assistance regarding Ethernet vs Infiniband IPoIB and lossless networks.

We have a 3 Node Windows 2016 Storage Spaces Direct Cluster that was setup early last year when documentation on S2D was still fairly sparse. We used Infiniband IPoIB instead of Ethernet because we have been using Infiniband for years to connect our Hyper-V Clusters to our Windows Storage SANs. The S2D setup is dis-aggregated, storage and hypervisors are separate, so the storage data and VM/Ethernet traffic are not over the same network.

We currently have a case open with Microsoft related to the Windows Server May Rollup which caused a problem with a VD after a server restart. The MS engineers have stressed that everything must be perfect in the networking creating a lossless network, including RoCE and QoS setup.

Since we are using IPoIB it has brought up the question is our configuration correct. Does Infiniband IPoIB provide provide lossless networking, needed for S2D traffic? And if so, is there anything that needs to be configured to achieve this?

Thanks,

Todd

Message was edited by: T Hunter

2

Hello Todd -

Please see: http://www.mellanox.com/related-docs/whitepapers/InfiniBandFAQ_FQ_100.pdf http://www.mellanox.com/related-docs/whitepapers/InfiniBandFAQ_FQ_100.pdf and let me know if it helps with your questions.

Specifically: 16 & 17

  • InfiniBand supports QoS by creating Virtual Lanes (VL). These VLs are separate logical communication links that share a single physical link.

  • InfiniBand, by contrast, uses link-level flow control to ensure that packets are not dropped in the fabric

Many thanks -

~Steve