ESXi 6 SRP initiator to Solaris COMSTAR SRP Target will crash ESXi 6 host with PSOD

1

SRP initiator 1.8.2.4 on ESXi 5.x show me a good RDMA performance.

But upgrade to ESXi 6 or above - include ESXi 6 update 2 with latest patch - then it show me a PSOD (Purple Screen Of Death) with APD (All Path Dead) randomly.

1027×770 118 KB

Why this problems occurs with COMSTAR target on Solaris families include OmniOS, Nexenta, etc.

IPoIB iSCSI also unstable.

I have a 2 of SX6036G VPI Gateway Switch then change all of MCX354A-FCBT’s operation mode to Ethernet with mlxconfig then connect to OmniOS IPoIB iSCSI Target via VPI Gateway.

I’m also heard that Mellanox will not support Infiniband iSER on ConnectX-3 or belows from some sources.

What’s your long term support policy?

If you drive Ethernet RoCE - RDMA Storage Protocol include - in virtualization then just say it!

I can’t remember how many times to wait for support RDMA storage protocols for every new ESXi.

Best Regards.

2

aviap Infrastructure & Networking - NVIDIA Developer Forums , can you, please, explain why iSER for ESXi is not available in IB mode for ANY Mellanox adaptor?

3

The incompatible drivers & platforms that you’re using seem to be the root cause of the PSOD

  • SRP initiator mlnx_ofed_esxi ver1.8.2.4 is not supporting ESXi 6.

  • It support Esxi5.X only, which proved to work fine in your case

  • As for iSER over RoCE v1 - ConnectX-3(10G 40G) is supported with Esxi6.and using mlnx eth driver v1.9.10.5, yet Solaris OS as a target was not tested therefore we cannot commit stable workability

Suggesting to approach Mellanox rlease note & user-manual to get more insight on the iSER supported oprtions

http://www.mellanox.com/page/products_dyn?product_family=29 http://www.mellanox.com/page/products_dyn?product_family=29

4

Okay!

I’m also try ETH driver 1.9.10.5 with my MCX354A-FCBT at ETH mode with mlnxconfig.

When I boot the ESXi 6.0u2 host with SX6036G ETH port mode shows me a 56Gb Ethernet link up, but vSphere network tab shows me a 10Gb link up…:(

What do you think about it?

Does it just display incorrectly unlike SX6036G’s MLNX-OS?

5

Hi!

I found some informant for new vSphere 6.5 native driver model.

VMware change driver model in vSphere 6.5.

That doesn’t permit to use vKernel driver like Mellanox old SRP, iSER for IB driver model.

That’s a big change, VMware will launch new ESXi 6.5 updates then support new driver that I can’t say it will be include iSER with IB.

But Mellanox said new ESXi update include SRIOV in IPoIB, but never said about iSER in IB or ETH mode.

VMware shows a very big bad hobby in historically.

That always change their product name and driver model.

That’s why Mellanox can’t support RDMA storage protocol on latest ESXi update.

VMware’s new native driver interact with ESXi hypervisor directly.

Therefore VMware must work to embedded iSER driver (IB or ETH) like TCP/IP iSCSI initiator.