path to OFED in Ubuntu 10.04

I installed OFED 1.5.4 on Ubuntu 10.04. Everything’s fine, ofed_info returns the list of installed components. However, I don’t know where OFED is installed when PGICDK ask me the location of OFED in order for it to install MVAPICH2. I did search for files/folder with ‘ofed’ in their names but I could not find any appropriate locations at all. Could you please give me some advices?

Also, as MvaPICH1,MvaPICH2,OpenMPI… were installed along with OFED 1.5.4, should I need to have MVAPICH2 built when install PGI CDK 11.10?

Thanks,
Tuan

Hi Tuan,

We don’t ship MVAPICH2 with the PGI CDK, I’m assuming you mean MVAPICH1.

I did search for files/folder with ‘ofed’ in their names but I could not find any appropriate locations at all.

Well, they could be installed any number of places. Locally we put ours in “/opt/ofed”. Can you go back to your OFED installation script to figure where it put them?

Also, as MvaPICH1,MvaPICH2,OpenMPI… were installed along with OFED 1.5.4, should I need to have MVAPICH2 built when install PGI CDK 11.10?

I think you mean MPICH1 and MPICH2. Though, they don’t use OFED so would be expected to install.

Is you interconnect Infiniband? You don’t need MVAPICH1 installed and only really want it if you have Infiniband.

  • Mat

Hi Mat,
Sorry, I thought it was MVAPICH2 as it just say MVAPICH for Infinibands. Yes I have a cluster of Mellanox IB adapters.
The following packages were installed with OFED 1.5.4 (the mvapich implementation).
$mpi-selector --list
mvapich-1.2.0
mvapich2-1.7
openmpi-1.4.3

Anyway, when I install PGI CDK, should MVAPICH be compiled to be able to use IB with PGI Fortran or I can use the packages already installed above? If so, is there any setting so that PGI Fortran recognize it.

Other wise, can you tell me which library PGI CDK needs to compile MVAPICH so that I can look for the locations of such library in my system; as I have no clue where to find where OFED was isntalled from the OFED install script (install.pl).


Thanks,
Tuan

Hi Tuan,

You don’t have to use the MPI libraries we ship. They are there for mostly for convenience with the added value of being able to access message queues from the PGI debugger. You certainly can use PGI compilers with MPI packages that you build yourself, including MVAPICH2.

The pre-built version of MVAPICH1 we ship does need to know the location of OpenFabric libraries. OFED is tied to your particular hardware and is why we can’t ship a pre-built version. I wish I could help you locate them on your system, but since they could be installed anywhere it will be up to you to find them. Perhaps if you reinstall them, you can view the output or logs? Note that even if you decide to build your own MVAPICH, you will need to know where OFED is.

  • Mat

Well, in fact MVAPICH1/2 were built automatically along with the installation of OFED using a single script. And the script is so complicated for me to detect which variable being used to specify the installation location. It would be easier if I know which libraries (.so files) that MVAPICH needs so that I can pull out the location easily. I’ll check the log though. Thanks Mat.

  • Tuan