I have installed PCGCDK 6.2 on my opteron cluster. On the compute nodes,
I have a Gb and an Ib network. On the master node, only a Gb interface.
Tests of latence and bandwitch are not good (130 us and 70Mb/s) compared
to another mpich installed on the cluster (2us and 900MB/s). I think that PGI
mpich use the Gb network and not the Infiniband one.
I am wondering if there is any configuration file I can modify to use the Ib network ? I have tried machines.LINUX without success.
The MPICH which is shipped with the CDK is configured for use with TCP since most clusters are able to support it. For Infiniband, you need to build your MPI library and link it with the software libraries which accompany your particular Infiniband vendor’s hardware. Two MPI implementation you can try are MVAPICH and OpenMPI. You might also want to check with your Infiniband vendor as to which MPI implementation they recommend.
Note that PGDBG, PGI’s MPI debugger, is currently only configured for use with MPICH and MPICH2 (as of the 7.0 release).
Hope this helps,