connect s1070 with InfiniBand

AMAX recommended us buy the Telsa s1070 with an InfiniBand, which actually is expensive. Moreover, I see that PCI express should be used to connect the s1070 to a host server in the Telsa specification. Anyone know about this InfiniBand? Or who has already used it before?

Whether you need inifiband probably comes down to how many CPU nodes are you buying and how often you are planning to run multi node MPI applications on your cluster. Infiniband has excellent latency characteristics, and you can achieve much better computational efficiency running MPI over Infiniband than TCP/IP over gigabit ethernet if you are using more than a couple of CPU nodes with your MPI code. But Infiniband switches are still rather expensive, so if you don’t have a lot of nodes, or you aren’t planning on using MPI with mulitple CPU nodes often, then maybe you could do without it.