RuntimeError: CUDA error: no kernel image is available for execution on the device

flyingalpaca · February 21, 2021, 10:56pm

I tried using synaptics and managed to remove the packages but ./deviceQuery is still showing cuda 10.0 instead of 11.0

Below is the output to dpkg -l |grep cuda

ii  cuda-11-0                                  11.0.3-1                                         amd64        CUDA 11.0 meta-package
ii  cuda-command-line-tools-11-0               11.0.3-1                                         amd64        CUDA command-line tools
ii  cuda-compat-11-0                           450.102.04-1                                     amd64        CUDA Compatibility Platform
ii  cuda-compiler-11-0                         11.0.3-1                                         amd64        CUDA compiler
ii  cuda-cudart-11-0                           11.0.221-1                                       amd64        CUDA Runtime native Libraries
ii  cuda-cudart-dev-11-0                       11.0.221-1                                       amd64        CUDA Runtime native dev links, headers
ii  cuda-cuobjdump-11-0                        11.0.221-1                                       amd64        CUDA cuobjdump
ii  cuda-cupti-11-0                            11.0.221-1                                       amd64        CUDA profiling tools runtime libs.
ii  cuda-cupti-dev-11-0                        11.0.221-1                                       amd64        CUDA profiling tools interface.
ii  cuda-demo-suite-11-0                       11.0.167-1                                       amd64        Demo suite for CUDA
ii  cuda-documentation-11-0                    11.0.228-1                                       amd64        CUDA documentation
ii  cuda-driver-dev-11-0                       11.0.221-1                                       amd64        CUDA Driver native dev stub library
ii  cuda-drivers                               460.32.03-1                                      amd64        CUDA Driver meta-package, branch-agnostic
ii  cuda-drivers-460                           460.32.03-1                                      amd64        CUDA Driver meta-package, branch-specific
ii  cuda-gdb-11-0                              11.0.221-1                                       amd64        CUDA-GDB
ii  cuda-gdb-src-11-0                          11.0.221-1                                       amd64        Contains the source code for cuda-gdb
ii  cuda-libraries-11-0                        11.0.3-1                                         amd64        CUDA Libraries 11.0 meta-package
ii  cuda-libraries-dev-11-0                    11.0.3-1                                         amd64        CUDA Libraries 11.0 development meta-package
ii  cuda-memcheck-11-0                         11.0.221-1                                       amd64        CUDA-MEMCHECK
ii  cuda-minimal-build-11-0                    11.0.3-1                                         amd64        Minimal CUDA 11.0 toolkit build packages.
ii  cuda-nsight-11-0                           11.0.221-1                                       amd64        CUDA nsight
ii  cuda-nsight-compute-11-0                   11.0.3-1                                         amd64        NVIDIA Nsight Compute
ii  cuda-nsight-systems-11-0                   11.0.3-1                                         amd64        NVIDIA Nsight Systems
ii  cuda-nvcc-11-0                             11.0.221-1                                       amd64        CUDA nvcc
ii  cuda-nvdisasm-11-0                         11.0.221-1                                       amd64        CUDA disassembler
ii  cuda-nvml-dev-11-0                         11.0.167-1                                       amd64        NVML native dev links, headers
ii  cuda-nvprof-11-0                           11.0.221-1                                       amd64        CUDA Profiler tools
ii  cuda-nvprune-11-0                          11.0.221-1                                       amd64        CUDA nvprune
ii  cuda-nvrtc-11-0                            11.0.221-1                                       amd64        NVRTC native runtime libraries
ii  cuda-nvrtc-dev-11-0                        11.0.221-1                                       amd64        NVRTC native dev links, headers
ii  cuda-nvtx-11-0                             11.0.167-1                                       amd64        NVIDIA Tools Extension
ii  cuda-nvvp-11-0                             11.0.221-1                                       amd64        CUDA Profiler tools
ii  cuda-runtime-11-0                          11.0.3-1                                         amd64        CUDA Runtime 11.0 meta-package
ii  cuda-samples-11-0                          11.0.221-1                                       amd64        CUDA example applications
ii  cuda-sanitizer-11-0                        11.0.221-1                                       amd64        CUDA Sanitizer
ii  cuda-toolkit-11-0                          11.0.3-1                                         amd64        CUDA Toolkit 11.0 meta-package
ii  cuda-tools-11-0                            11.0.3-1                                         amd64        CUDA Tools meta-package
ii  cuda-visual-tools-11-0                     11.0.3-1                                         amd64        CUDA visual tools

generix · February 21, 2021, 11:19pm

Did you reboot?

flyingalpaca · February 21, 2021, 11:25pm

yes

generix · February 21, 2021, 11:28pm

Please post the output of
ldd ./deviceQuery

flyingalpaca · February 21, 2021, 11:35pm

	linux-vdso.so.1 (0x00007fff489dc000)
	librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007f1b073c6000)
	libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f1b071a7000)
	libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f1b06fa3000)
	libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007f1b06c1a000)
	libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f1b06a02000)
	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f1b06611000)
	/lib64/ld-linux-x86-64.so.2 (0x00007f1b07854000)

generix · February 22, 2021, 8:58am

Looking at deviceQuery output

CUDA Device Query (Runtime API) version (CUDART static linking)

can you do a rebuild of deviceQuery, since it’s statically linked.

flyingalpaca · February 22, 2021, 7:59pm

Hello,

I tried doing make clean to allow me to rebuild the binaries.
After that, when I try make again, I get the following error

/usr/local/cuda-10.0/bin/nvcc -ccbin g++ -I../../common/inc  -m64    -gencode arch=compute_30,code=sm_30 -gencode arch=compute_35,code=sm_35 -gencode arch=compute_37,code=sm_37 -gencode arch=compute_50,code=sm_50 -gencode arch=compute_52,code=sm_52 -gencode arch=compute_60,code=sm_60 -gencode arch=compute_61,code=sm_61 -gencode arch=compute_70,code=sm_70 -gencode arch=compute_75,code=sm_75 -gencode arch=compute_75,code=compute_75 -o deviceQuery.o -c deviceQuery.cpp
make: /usr/local/cuda-10.0/bin/nvcc: Command not found
Makefile:288: recipe for target 'deviceQuery.o' failed
make: *** [deviceQuery.o] Error 127

It seems to be looking for something in the cuda-10.0 directory, which no longer exists cause I uninstalled cuda 10.0

Mart · February 22, 2021, 9:04pm

Is there a configure script? You should run that again before doing the make.

flyingalpaca · February 22, 2021, 10:50pm

Hello,
I’m quite new to all this so I’m not too sure what you mean by the configure script.
In the directory where the deviceQuery makefile is, there are only the following files:

Also, in the Makefile, the first line has

# Location of the CUDA Toolkit                                                                       
CUDA_PATH ?= /usr/local/cuda-11.0

so I’m not too sure why its looking in cuda-10.0 in the first place

generix · February 22, 2021, 11:26pm

Maybe CUDA_PATH is set in your env?
export |grep CUDA