pycuda Error: “nvcc fatal : Value 'sm_61' is not defined for option 'gpu-architecture”

I am using Ubuntu 16.04 and I am trying to execute this code:

import pycuda.driver as cuda
import pycuda.autoinit
from pycuda.compiler import SourceModule

import numpy
a = numpy.random.randn(4,4)

a = a.astype(numpy.float32)

a_gpu = cuda.mem_alloc(a.nbytes)

cuda.memcpy_htod(a_gpu, a)

mod = SourceModule("""
  __global__ void doublify(float *a)
  {
    int idx = threadIdx.x + threadIdx.y*4;
    a[idx] *= 2;
  }
  """)

I got the error:

pycuda.driver.CompileError: nvcc compilation of /tmp/tmpzohid21m/kernel.cu failed
[command: nvcc --cubin -arch sm_61 -I/usr/local/lib/python3.5/dist-packages/pycuda/cuda kernel.cu]
[stderr:
nvcc fatal : Value ‘sm_61’ is not defined for option ‘gpu-architecture’
]

Any idea?

You have a version of CUDA that precedes 8.0

This would be the version that pycuda is using.

I installed using apt get install cuda

What should I do to solve this? I am newbie.

I think you probably have a mixed up install. I assume this is your cross-posting:

https://stackoverflow.com/questions/52029350/pycuda-error-nvcc-fatal-value-sm-61-is-not-defined-for-option-gpu-archit

What is the result of running:

nvcc --version

and

sudo find / -name nvcc

and

echo $PATH

and

echo $LD_LIBRARY_PATH

?

nvcc --version
nvcc: NVIDIA ® Cuda compiler driver
Copyright © 2005-2015 NVIDIA Corporation
Built on Tue_Aug_11_14:27:32_CDT_2015
Cuda compilation tools, release 7.5, V7.5.17


sudo find / -name nvcc
/usr/lib/nvidia-cuda-toolkit/bin/nvcc
/usr/bin/nvcc
/usr/local/cuda-9.2/bin/nvcc
find: ‘/run/user/1000/gvfs’: Permission denied


echo $PATH
/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin:/usr/lib/jvm/java-8-oracle/bin:/usr/lib/jvm/java-8-oracle/db/bin:/usr/lib/jvm/java-8-oracle/jre/bin:/usr/local/cuda/bin


echo $LD_LIBRARY_PATH

nothing!


This is a problem:

Cuda compilation tools, release 7.5, V7.5.17

CUDA 7.5 is being used. It doesn’t support sm_61

You have multiple CUDA versions installed on your system:

/usr/bin/nvcc
/usr/local/cuda-9.2/bin/nvcc

And the one in /usr/bin is the one being used and it is too old for sm_61

You need to clean that system up. The CUDA version you intend to use should be the first one that is found by working through your PATH variable:

/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin:/usr/lib/jvm/java-8-oracle/bin:/usr/lib/jvm/java-8-oracle/db/bin:/usr/lib/jvm/java-8-oracle/jre/bin:/usr/local/cuda/bin

As we go through that PATH variable, we hit /usr/bin and there is a /usr/bin/nvcc on your system, and its not the one you want to use.

One suggestion would be to remove all CUDA versions on your system except the one you intend to use.

Another possibility would be to make sure the PATH variable points (first) at the CUDA version you intend to use.

In any event it seems you didn’t follow the install instructions correctly. Pay particular attention to section 7:

https://docs.nvidia.com/cuda/cuda-installation-guide-linux/index.html#post-installation-actions

You haven’t performed section 7.1.1 correctly.

I changed the variables:

nvcc --version
nvcc: NVIDIA ® Cuda compiler driver
Copyright © 2005-2018 NVIDIA Corporation
Built on Tue_Jun_12_23:07:04_CDT_2018
Cuda compilation tools, release 9.2, V9.2.148

sudo find / -name nvcc
/usr/lib/nvidia-cuda-toolkit/bin/nvcc
/usr/bin/nvcc
/usr/local/cuda-9.2/bin/nvcc
find: ‘/run/user/1000/gvfs’: Permission denied


echo $PATH
/usr/local/cuda-9.2/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin:/usr/lib/jvm/java-8-oracle/bin:/usr/lib/jvm/java-8-oracle/db/bin:/usr/lib/jvm/java-8-oracle/jre/bin:/usr/local/cuda/bin


echo $LD_LIBRARY_PATH
/usr/local/cuda-9.2/lib64

But it is still not working. What should I do?
Thanks!

pycuda is probably not using your PATH variable

If it were my system, I would start over with a clean install of the linux OS, and install just CUDA 9.2

You could also try just removing the CUDA 7.5 nvcc:

sudo rm -f /usr/bin/nvcc

I have followed this:

https://developer.nvidia.com/cuda-downloads?target_os=Linux&target_arch=x86_64&target_distro=Ubuntu&target_version=1604&target_type=deblocal

I did apt-get install cuda

Now, I got the error:

pytools.prefork.ExecError: error invoking 'nvcc --preprocess -arch sm_61 -I/usr/local/lib/python3.6/dist-packages/pycuda/cuda /tmp/tmp6iq4ku3w.cu --compiler-options -P': [Errno 2] No such file or directory: 'nvcc'

What is going on?

When I run the code using terminal:

python3.6 t7.py

I got no error

When I run the same code using an IDE (like pycharm):

I got the error:

pytools.prefork.ExecError: error invoking 'nvcc --version': [Errno 2] No such file or directory: 'nvcc'

What is wrong?

The IDE is not picking up your environment variables. You’ll need to research how to make that work in pycharm.

maybe start here:

https://stackoverflow.com/questions/21538859/pycharm-set-environment-variable-for-run-manage-py-task

/usr/local/cuda-9.2/lib64\${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}

and

/usr/local/cuda-9.2/bin${PATH:+:${PATH}}

to pycharm but I got the same error

I tried configure variable enviroment in Wing Python IDE too:

https://imgur.com/qFwg4Ab

https://imgur.com/UOqomMo

But it is not working.

I only can run the code using terminal!