PyCUDA ImportError (Ubuntu and Cuda 9.1)

My goal is to speed up my python FEA (finite elements analysis) with my quadro GPU. I however have issues when I import pycuda.autoinit or pycuda.driver into my python code. See the example from my Console:

In [6] import pycuda.autoinit
Traceback (most recent call last):

  File "<ipython-input-7-78816ba4a0fc>", line 1, in <module>
    import pycuda.autoinit

  File "/home/bram/.anaconda3/lib/python3.6/site-packages/pycuda-2017.1-py3.6-linux-x86_64.egg/pycuda/autoinit.py", line 2, in <module>
    import pycuda.driver as cuda

  File "/home/bram/.anaconda3/lib/python3.6/site-packages/pycuda-2017.1-py3.6-linux-x86_64.egg/pycuda/driver.py", line 5, in <module>
    from pycuda._driver import *  # noqa

ImportError: libcurand.so.8.0: cannot open shared object file: No such file or directory

Some details of my setup:

  • HP Zbook Studio G3 (Quadro M1000M) Ubuntu 18.04
  • Cuda 9.1 (.run installer)(I added the path variables to ~/.bashrc
  • nvidia-driver-390 as driver
  • pycuda 2017.1 (from anaconda)

I’ve tried the solutions proposed by people encountering similar issues when using tensorflow-gpu: It was proposed to make a softlink from libcurand.se.8.0 to the libcurand.se.9.1 using the terminal:

user@device:~$ sudo ln -s libcublas.so.9.1 libcublas.so.8.0

This did not help however.

I’ve checked the installation of CUDA by running a simple vectorAdd example in Exlips. That worked without any issues and when profiling it showed that the gpu was working as expected.

I probably made a mistake somewhere and tell me if you need more information

Your pycuda install is expecting CUDA 8. You cannot use CUDA 9 or 9.1 in any way as a substitute for CUDA 8.

You can install CUDA 8 “along side” CUDA 9/9.1, fairly easy with the runfile install method.

Thanks for your swift response, I was not aware that one could install the two side by side and just tried it. Sadly the instalation failed.

Attempt 1: returned the following log:

Error: unsupported compiler: 7.3.0. Use --override to override this check.

===========
= Summary =

Driver: Not Selected
Toolkit: Installation Failed. Using unsupported Compiler.
Samples: Not Selected

Logfile is /tmp/cuda_install_9615.log

Attempt 2: Now using the -override option returned the following log:

Installing the CUDA Toolkit in /usr/local/cuda-8.0 …
Verifying archive integrity… All good.
Uncompressing NVIDIA CUDA… (dots go on fo a long time)

Can’t locate InstallUtils.pm in @INC (you may need to install the InstallUtils module) (@INC contains: /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.26.1 /usr/local/share/perl/5.26.1 /usr/lib/x86_64-linux-gnu/perl5/5.26 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.26 /usr/share/perl/5.26 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base) at ./install-linux.pl line 6.

BEGIN failed–compilation aborted at ./install-linux.pl line 6.

Verifying archive integrity… All good.

Uncompressing NVIDIA CUDA Samples… (dots go on fo a long time)

Can’t locate InstallUtils.pm in @INC (you may need to install the InstallUtils module) (@INC contains: /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.26.1 /usr/local/share/perl/5.26.1 /usr/lib/x86_64-linux-gnu/perl5/5.26 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.26 /usr/share/perl/5.26 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base) at ./install-sdk-linux.pl line 6.

BEGIN failed–compilation aborted at ./install-sdk-linux.pl line 6.

‘uninstall_cuda_8.0.pl’ -> ‘/usr/local/cuda-8.0/bin/uninstall_cuda_8.0.pl’

===========
= Summary =

Driver: Not Selected
Toolkit: Installation Failed
Samples: Not Selected

Logfile is /tmp/cuda_install_9709.log

I’m not sure what is going wrong in the second attempt

https://devtalk.nvidia.com/default/topic/983777/cuda-setup-and-installation/can-t-locate-installutils-pm-in-inc/

I looked at the errors and had a look at the forum. see https://devtalk.nvidia.com/default/topic/983777/cuda-setup-and-installation/can-t-locate-installutils-pm-in-inc/ This worked. It did not solve my issue however.

(I did edit my path variables as well

user@device:~$ which nvcc 
/usr/local/cuda-8.0/bin/nvcc

The error that python returned to me is the same as it was before

Just editing your PATH variable is not enough.

You have to add the /usr/local/cuda-8.0/lib64 to the LD_LIBRARY_PATH variable.

I have added the following to my .bashrc file:

export LD_LIBRARY_PATH=/usr/local/cuda-8.0/lib64${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}

did you log out and log in or otherwise make sure that .bashrc file is sourced in your current environment?

stated another way, what is the result of:

echo $LD_LIBRARY_PATH

?

I get the following:

user@device:~$ echo $LD_LIBRARY_PATH
/usr/local/cuda-8.0/lib64:/usr/local/cuda/extras/CUPTI/lib64:/usr/local/cuda-9.1/lib64

Thanks for your swift answers

You should confirm that

/usr/local/cuda-8.0/lib64/libcurand.so.8.0

exists. If it exists, then your python is not picking up your environment for some reason.

I just checked it, the file libcurand.so.8.0 does exist. I’ll have a look on what could be wrong at the python side later tonight.

thanks for all the help

python only parses the LD_LIBRARY_PATH variable when the python interpreter starts. So if you are leaving your python session open throughout all this, that won’t work.

Try logging out entirely. Or rebooting the machine.

Yeah, I did reboot. I’ve made that mistake a previous time.

I’ve got it, I needed to install the python cudatoolkit-8.0 version as well! I’m so sorry for wasting your time :-(