Build a Mellanox Infrastructure on ESXi 6.5U1

Infrastructure & Networking Software And Drivers Virtualization For Infiniband And Ethernet
1

Hi!

I’m test Mellanox solutions that include Mellanox MCX354A-FCBT ConnectX-3 HCAs with firmware 2.42.5000, Mellanox FDR14 2m copper cables, Mellanox SX6036G Gateway switches with firmware 3.6.4006 on vSphere 6.5 update 1 for POC.

It’ll be going with 3 steps like belows.

01. Build a initial network connection with TCP/IP (lossy) network

02. Build a PFC network

03. Build a iSER RDMA Storage

Here are POC systems lists.

* ESXi hosts

: 5 x Fujitsu RX200 S7 ESXi Hosts that include 2Way E5-2670, 128GB Memory, RAID1 SSD boot disk for ESXi, Mellanox ConnectX-3 MCX354A-FCBT

* Ethernet Switch

: 2 x Mellanox SX6036G Gateway Systems as a PFC Ethernet switch that support 1, 10, 40, 56Gb Ethernet port mode

* Storage

: 5 x SuperMicro Custom Storage with ESOS SCST Target - 1Way E5-2660, 64GB Memory, Mellanox ConnectX-3 MC354A-FCBT

I want to build PFC based infrastructures that include

01.General purpose VM ETH network with TC0 that lossy ETH network

  1. Build a PFC ETH network

02-A. PFC based iSER RDMA Storage ETH network with TC1 that loseless ETH network

03-B. PFC based vMotion RDMA ETH network with TC02 that leseless ETH network

  1. General purpose vKernel NFS network for NFS Storage with TC04 that lossy ETH network

At first, some times later I’ll post step 01 results.

Anybody who interesting this POC test, participate this threads…

BR,

Jae-Hoon Choi

2

I gave up this project.

Mellanox deprecate ConnectX-3 & Connect-3 Pro products.

ConnectX-3 will be deprecate on 1 Dec., 2017.

ConnectX-3 Pro will be deprecate on 31 Jan., 2018.

719×892 33.4 KB

I predict new iSER driver on ESXi 6.5 & next relase ESXi 6.7 will be support ConnectX-4 or above products.

Regards,

Jae-Hoon Choi

P.S

ConnectX-4 OCP product also deprecated.

In worst case, I think move to ConnectX-5 or ConnectX-6 product.