In-Cast or Micro Burst on SN2000 Series Switch

Infrastructure & Networking Adapters and Cables Ethernet Adapter Cards
1

Hi …,

I would like to know whether a 100Gbps NIC is able to achieve full 100Gbps speeds, without RoCE / RDMA / VMA.

Consider a compute farm like scenario.

Say, there is a NAS with a 100Gps NIC, connected to an SN2100 Switch.

The 100Gbps switch is in turn connected to 4 x 48 Port 1Gbps switches using a 40G links to each switch.

i.e. 192 x 1Gbps client computers

|----------| |-----------| —40G—> | 48 Port 1G Switch |

| NAS | —100G NIC —> |SN2100| —40G—> | 48 Port 1G Switch |

|Storage| | Switch | —40G—> | 48 Port 1G Switch |

|----------| |-----------| —40G—> | 48 Port 1G Switch |

Now, if all the 192 x 1Gbps clients were to read files from the storage at the same time, will the NAS NIC be able to serve at 100Gbps (assuming that there are no bottlenecks in the storage system itself)?

Regards,

Indivar Nair

2

With no RoCE / RDMA / VMA and as per the configuration you’ve presented - then theoraticaly, the NAS nic of the Storage node is able to server 100Gb/s only practically the switch is likely to cause traffic-congestion In more details, when at the same time a total of 192Gb/s traffic is tunneled through 160Gb/s switch (4x 40Gb), hitting an adapter with "only "100Gb/s capability - traffic will go into congestion state and will drop down drastically due to switch running into “buffer overflow”.

A workaround will be setting Flow-control (Paused-Frames" on adapter interface & switch ports).