I installed a pgi workstation with c/c++/fortran on a cluster but locally within my own account. I had both cuda and mpich1 installed. I also set the environment variables (like PATH, LIBRARY_PATH, and LD_LIBRARY_PATH) according to the install.txt.
Then I download the source code from this website (Account Login | PGI) and compile them. The CPU version and ACC version comiples and runs well, but the CUDA version can’t compile, with the error message:
“/usr/bin/ld: cannot find -lcuda.”
Given that the makefile is somewhat complex, I tried a single file and compile it with:
pgfortran -Mcuda(or cuda5.0) filename
but the error is still there.
I guess it’s because the linker can’t find the library “libcuda.so”. I then looked for the file in all the lib/, lib64/, libso/ files that I can think of, however, I can only find something like “libcudafor.so”, “libcudafor4.so”, “libcudafor4.a” etc. I even tried to create a symbolic link “libcuda.so” to “libcudafor.so”, this time the link error goes way, but there are other errors like “__rangdomgpu_entry” unresolved/unreference etc. (don’t remember exactly the error message), and I guess it’s because I am linking to the wrong library.
So, my question is, why is this error happening? BTW, I am running on Linux (CentOS release 6.3 (Final) – just realized this weird distro) with x86_64.
Thanks,
Sheng
After playing a little bit, it seems that if I make a symbolic of “libcuda.a” in pgi/linux86-64/13.4/lib to “libcudafor.a” in the same directory, the code can compile and runs fine. There are still errors when compiling the last program,
undefined reference to `randomgpu__entry’,
which I mentioned before, but it seems to be a C binding problem. Also, the environment variable, LIBRARY_PATH and LD_LIBRARY_PATH, don’t seem to be necessary for the code to run.
So what might be the reason that I have to do this tweak in order to make the compiler work? In addition, is “libcudafor.a” the right library to link to (there is also a “libcudafor4.a” in the same directory)?
Thanks,
Sheng
Hi Sheng,
“libcuda.so” is part of your CUDA driver installation so making “libcuda.a” a symbolic link to “libcudafor.a” isn’t the solution. The “libcuda.so” library should be located in “/usr/lib64”. Though, since you’ve installed CUDA locally, you may need to use “-L” flag to point the linker to the correct directory.
undefined reference to `randomgpu__entry’,
NVCC periodically changes their name mangling so you will need to use “nm” on the CUDA object to determine what the actual symbol name is. Most likely, they’ve dropped the “__entry”.
Hope this helps,
Mat
Hi Mat,
Thanks a lot for your reply. I just saw this – I thought there might be some sort of notification sent to my email.
In the cluster there was a CUDA installation which was mainly for NVCC (not sure if it’s compatible with pgfortran). However, in the main node there is no such lib file as /usr/lib64/libcuda.so, and it only exists on the node where there is GPU (maybe the admin only installs the driver on the nodes with GPU)
When I install the pgfortran suite, I also installed the CUDA and MPI along with it, and added the path to the environment variable.
So if now I compile a program in NVCC and use ldd to the executable, this is what I get:
linux-vdso.so.1 => (0x00007fffa6c3c000)
libcudart.so.5.0 => /my_home_directory/pgi/linux86-64/2013/cuda/5.0/lib64/libcudart.so.5.0 (0x00007fe47a43f000)
libstdc++.so.6 => /usr/lib64/libstdc++.so.6 (0x000000378d200000)
libm.so.6 => /lib64/libm.so.6 (0x0000003782600000)
libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x0000003787e00000)
libc.so.6 => /lib64/libc.so.6 (0x0000003781a00000)
libpthread.so.0 => /lib64/libpthread.so.0 (0x0000003782200000)
libdl.so.2 => /lib64/libdl.so.2 (0x0000003781e00000)
librt.so.1 => /lib64/librt.so.1 (0x0000003782a00000)
/lib64/ld-linux-x86-64.so.2 (0x0000003781600000)
which means that NVCC is not using libcuda.so here.
If I copy the libcuda.so file from where it exists to the main node, and use -L explicitely when compiling the cuda fortran code, it works. But when I just add it to the lib directory where the pgfortran lib files are (and this directory is included in both LIBRARY_PATH and LD_LIBRARY_PATH), without using the -L flag, it doesn’t work.
So here are my two questions: 1) why does NVCC not need libcuda.so during compiling and linking but pgfortran needs it? 2) why do I need to use -L explicitely in the second case?
Thanks,
Sheng
why does NVCC not need libcuda.so during compiling and linking but pgfortran needs it?
In 13.1 we started adding libcuda to the link with CUDA Fortran. However, we’re revisiting this for 13.6 (TRP#19114)
why do I need to use -L explicitely in the second case?
If the library was put into a directory already in the standard library path, has read permissions, and you’re not statically linking, then the linker should have been able to find it. I’m not sure why it didn’t for you.