What are the recommended IB configuration for large MPI jobs in a cluster having more than 1000 nodes?

Infrastructure & Networking Software And Drivers Mellanox OFED
1

Hi,

System details are,

IB HCA- MT26428 [ConnectX VPI PCIe 2.0 5GT/s & MT27500 Family [ConnectX-3]

OFED Version: MLNX_OFED_LINUX-1.5.3-3.0.0

RHEL 5.8 kernel**: 2.6.18-308.el5**

Thanks.