cudaErrorInsufficientDriver but everything seems to be fine

Accelerated Computing CUDA CUDA Programming and Performance
1

I have correctly set device number and cuda paths for different devices on different systems. However, I don’t know what is wrong with the following error:

CUDA error at euler3d.cu:433 code=35(cudaErrorInsufficientDriver) “cudaSetDevice(0)”

The build command is

$ make
nvcc  -O2 -Xptxas -v --gpu-architecture=compute_61 --gpu-code=compute_61 euler3d.cu -o euler3d -I/usr/local/cuda-10.0/samples/common/inc  -L/usr/local/cuda-10.0/samples/lib
nvcc -Xptxas -v -O3 --gpu-architecture=compute_61 --gpu-code=compute_61 euler3d_double.cu -o euler3d_double -I/usr/local/cuda-10.0/samples/common/inc  -L/usr/local/cuda-10.0/samples/lib
nvcc -Xptxas -v -O3 --gpu-architecture=compute_61 --gpu-code=compute_61 pre_euler3d.cu -o pre_euler3d -I/usr/local/cuda-10.0/samples/common/inc  -L/usr/local/cuda-10.0/samples/lib
pre_euler3d.cu:36:2: warning: #warning "the kernels may fail too launch on some systems if the block length is too large" [-Wcpp]
 #warning "the kernels may fail too launch on some systems if the block length is too large"
  ^
pre_euler3d.cu:36:2: warning: #warning "the kernels may fail too launch on some systems if the block length is too large" [-Wcpp]
 #warning "the kernels may fail too launch on some systems if the block length is too large"
  ^
nvcc -Xptxas -v -O3 --gpu-architecture=compute_61 --gpu-code=compute_61 pre_euler3d_double.cu -o pre_euler3d_double -I/usr/local/cuda-10.0/samples/common/inc  -L/usr/local/cuda-10.0/samples/lib

nvidia-smi and CUDA_VISIBLE_DEVICES are also fine.

$ echo $CUDA_VISIBLE_DEVICES
0
$ nvidia-smi
Sat Nov 30 18:26:30 2019
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 410.48                 Driver Version: 410.48                    |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|===============================+======================+======================|
|   0  GeForce GTX 108...  Off  | 00000000:02:00.0 Off |                  N/A |
|  0%   42C    P8    14W / 250W |      0MiB / 11178MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+
|   1  GeForce GTX 980 Ti  Off  | 00000000:81:00.0 Off |                  N/A |
| 44%   41C    P8    15W / 250W |      0MiB /  6083MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes:                                                       GPU Memory |
|  GPU       PID   Type   Process name                             Usage      |
|=============================================================================|
|  No running processes found                                                 |
+-----------------------------------------------------------------------------+

So, I don’t know who is causing the problem that the program may think there is another driver or so!
Any comment?

2

before you do:

$ make

what is the result of running

nvcc --version

?

3

Thank you very much for the hint.

$ nvcc --version
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2019 NVIDIA Corporation
Built on Wed_Apr_24_19:10:27_PDT_2019
Cuda compilation tools, release 10.1, V10.1.168

And it seems that nvcc is used from the PATH variable. I fixed that with /usr/local/cuda-10.0/bin/nvcc to match with includes and libraries.