How to link with MKL?

Accelerated Computing HPC Compilers Legacy PGI Compilers
1

I have found the PGI included (reference) Blas to be as slow as expected, so I am trying to use the Intel MKL. However, I cannot figure out how to link these libraries with PGI Workstation 10.3. This is in the Windows 7 64 bit environment.

Here is the source file (zoo.f95) I am using:

program zoo
real(8), dimension(:,:), allocatable :: x, y
real(8), u, v
character(len=64) arg
integer i, j, k, n
call get_command_argument(1, arg)

read(arg, ‘(I5)’) n
allocate(x(n,n), y(n,n))

! fill the x array with the Hilbert matrix
do i = 1, n
x(i,:) = dble(i - 1)
end do
do j = 1, n
v = dble(j)
x(:,j) = 1.0d0 / (x(:,j) + v)
end do
y = 0.0d0

! equivalent to y = matmul(x, x)
call dgemm(‘n’, ‘n’, n, n, n, 1.0d0, x, n, x, n, 0.0d0, y, n)

! output the sum to ensure result was calculated
print *, sum(y)
end

According to bash, the apparent location of the MKL libraries on my system is:

/c/Program Files/Intel/MKL/10.2.4.032/em64t/lib

However the space in the pathname necessitates quotes of some sort.

I have tried several things. At the moment I am trying

pgfortran -fast -Minfo -Mconcur zoo.f95 -L/c/“Program Files”/Intel/MKL/10.2.4.032/em64t/lib -lmkl_core

but this does not seem to work.

I get the error:

LINK : fatal error LNK1004: cannot open file libmkl_core.lib

The first issue here is that the MKL library files do not have names that start with lib. Surely there is a way to let the linker in on the joke - in fact I have a very dim memory of needing to do this once many years ago. The point is, how to tell the PGI linker about this.

The second issue is that I am not convinced the linker is even looking in the first place. There is, in the directory in question, a file called libguide.lib. So when I attempt to compile with

pgfortran -fast -Minfo -Mconcur zoo.f95 -L/c/“Program Files”/Intel/MKL/10.2.4.032/em64t/lib -lguide

and I get the error

LINK : fatal error LNK1004: cannot open file libguide.lib

which does not bode well.

I have been through the user’s guides and manuals for PGI and MKL but haven’t come across a hint for this.

Anyone here know how to link with MKL on Windows 7 64 bit?

Alternatively, is there a freely available BLAS/LAPACK which I can compile with PGI that is roughly performance comparable to MKL for intel CPUs?

2

Hello zenpharoahs. If you go to the latest release of MKL you will see that they now support compilation with PGI, albeit 9.3. Nevertheless it does compile nicely with 10.3.

There is still a problem of find the correct libraries when compiling and linking an application. Maybe you can find them.

It would be very helpful if someone from PGI would take a look at this!

Malcolm

3

Hi zenpharaohs,

Compilers prior to 10.4 need to use DOS path names. Change “-L/c/” to “-LC:/” and the compiler will be able to find the libguide.lib library.

“-lname” will translate the name of the library into the form “libname.lib”. For libraries not in this form, add the full name of the library to the link line, not including “-l” or “-L”. For example: “C:/Progra~1/Intel/MKL/10.2.4.032/em64t/mkl_core.lib”.

Hope this helps,
Mat

4

Well this got me to the point where PGI linker is looking in that directory and grabbing symbols, but I will have to figure out which libraries to specify; it’s not obvious, and the combinations I have used either don’t cover all the symbols which need resolving, or else have multiple conflicting definitions of symbols. This is my first time using MKL in many years, and it seems that they have several layers of complexity in the library structure since then. I would have hoped for something like “-lmkl”.

I suppose I can try building a custom library (intel provides some sort of tool for this) but it’s getting to be a pretty long walk just to be able to see whether the intel MKL dgemm is faster or not.

5

zenpharaohs,

I don’t know if it helps with your issue, but I’ve found when using MKL nowadays with the dizzying array of possibilities, that the Intel MKL Link Line Advisor is nigh-mandatory.

It’s not always perfect, but it can give you an idea about the correct order for the link line which is quite touchy with MKL (in my experience).

6

OK I have “made it work”. The intel MKL link advisor tip was very helpful, but the real trick is that you just cannot tell the linker to look for a pathname with a space in it. There are supposed to be various tricks to make this go, and at various times I thought I was getting somewhere with that. There are also supposed to be those Windows tilde-encrusted nonspace pathnames, but I didn’t build any executables that way. Yet.

But really just put all the library files where you can refer to them with a pathname with no spaces in it. Then things work just fine. Life is too short to figure out demented pathnames.

And the results are in on the Nehalem-64 machine I am using here (four cores with HT). Even though I am a bit clever at this sort of thing, and the PGI compiler is actually pretty good, the Fortran DGEMM I wrote for myself to use while figuring this nonsense out is only the same speed as using the ACML (although AMD centric it at least works) - and the intel MKL is twice as fast. Getting relatively mildly optimized Fortran to speed within a factor of two of MKL (which I suspect is optimized like a house on fire) actually speaks pretty well of the compiler. However it is quite strange how MKL gets the top performance - it uses only four of the eight available hyperthreads, I expect it defeats hyperthreading. But the PGI compiled code uses all eight hyperthreads to get best performance - reducing the NCPUS from 8 to 4 does reduce the number of cores used by the PGI generated code, but is also slower than the obvious choice 8. Even setting NCPUS to 9 is faster than setting it to 4.

One annoying point is that the matmul primitive in Fortran does not compile to anything fast even when linking with the intel MKL - even if the command line includes the MKL libraries explicitly before the program containing the matmul call. It would also have been nice if the compiler knew to translate idioms like matmul(transpose(x), x), etc., into xGEMM calls (since xGEMM knows how to do this). In any case, I know how to translate these idioms, and at least I know I should not leave it up to the compiler. This is a pretty disappointing, since Sun was touting that their compiler knew how to do this in 2005 (http://blogs.sun.com/hinkthink/entry/the_fortran_compiler_f95_and). Is there a way to explicitly tell the compiler to do this besides writing xGEMM calls in the source?