CUDA deviceQuery does not work but deviceQueryDrv does

Accelerated Computing CUDA CUDA Setup and Installation
1

I’m following the CUDA installation guide and at step 2.5.1. Running the Compiled Examples you have to compile and run deviceQuery.exe to see if CUDA is installed and configured correctly.

I can compile deviceQuery without any error or warnings, but when I run it I get:

C:\ProgramData\NVIDIA Corporation\CUDA Samples\v11.2\bin\win64\Debug\deviceQuery.exe Starting...

 CUDA Device Query (Runtime API) version (CUDART static linking)

cudaGetDeviceCount returned 3
-> initialization error
Result = FAIL

However, if I compile and run deviceQueryDrv.exe I get the expected result:

C:\ProgramData\NVIDIA Corporation\CUDA Samples\v11.2\bin\win64\Debug\deviceQueryDrv.exe Starting...

CUDA Device Query (Driver API) statically linked version
Detected 1 CUDA Capable device(s)

Device 0: "Quadro K610M"
  CUDA Driver Version:                           10.1
  CUDA Capability Major/Minor version number:    3.5
  Total amount of global memory:                 1024 MBytes (1073741824 bytes)
  ( 1) Multiprocessors, (192) CUDA Cores/MP:     192 CUDA Cores
  GPU Max Clock rate:                            954 MHz (0.95 GHz)
  Memory Clock rate:                             1300 Mhz
  Memory Bus Width:                              64-bit
  L2 Cache Size:                                 524288 bytes
  Max Texture Dimension Sizes                    1D=(65536) 2D=(65536, 65536) 3D=(4096, 4096, 4096)
  Maximum Layered 1D Texture Size, (num) layers  1D=(16384), 2048 layers
  Maximum Layered 2D Texture Size, (num) layers  2D=(16384, 16384), 2048 layers
  Total amount of constant memory:               65536 bytes
  Total amount of shared memory per block:       49152 bytes
  Total number of registers available per block: 65536
  Warp size:                                     32
  Maximum number of threads per multiprocessor:  2048
  Maximum number of threads per block:           1024
  Max dimension size of a thread block (x,y,z): (1024, 1024, 64)
  Max dimension size of a grid size (x,y,z):    (2147483647, 65535, 65535)
  Texture alignment:                             512 bytes
  Maximum memory pitch:                          2147483647 bytes
  Concurrent copy and kernel execution:          Yes with 1 copy engine(s)
  Run time limit on kernels:                     Yes
  Integrated GPU sharing Host Memory:            No
  Support host page-locked memory mapping:       Yes
  Concurrent kernel execution:                   Yes
  Alignment requirement for Surfaces:            Yes
  Device has ECC support:                        Disabled
  CUDA Device Driver Mode (TCC or WDDM):         WDDM (Windows Display Driver Model)
  Device supports Unified Addressing (UVA):      Yes
  Device supports Managed Memory:                Yes
  Device supports Compute Preemption:            No
  Supports Cooperative Kernel Launch:            No
  Supports MultiDevice Co-op Kernel Launch:      No
  Device PCI Domain ID / Bus ID / location ID:   0 / 1 / 0
  Compute Mode:
     < Default (multiple host threads can use ::cudaSetDevice() with device simultaneously) >
Result = PASS

I searched the forum but could not find any similar problems where deviceQueryDrv was working and deviceQuery was not.

Further information about the software and GPU after running nvcc -V

nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2021 NVIDIA Corporation
Built on Thu_Jan_28_19:41:49_Pacific_Standard_Time_2021
Cuda compilation tools, release 11.2, V11.2.142
Build cuda_11.2.r11.2/compiler.29558016_0

and after nvidia-smi.exe

+-----------------------------------------------------------------------------+
| NVIDIA-SMI 426.50       Driver Version: 426.50       CUDA Version: 10.1     |
|-------------------------------+----------------------+----------------------+
| GPU  Name            TCC/WDDM | Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|===============================+======================+======================|
|   0  Quadro K610M       WDDM  | 00000000:01:00.0 Off |                  N/A |
| N/A   41C    P8    N/A /  N/A |    155MiB /  1024MiB |     17%      Default |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes:                                                       GPU Memory |
|  GPU       PID   Type   Process name                             Usage      |
|=============================================================================|
|    0                    Not Supported                                       |
+-----------------------------------------------------------------------------+
2

I can report exactly the same observation, Windows 10, VS 2019, NVS 310 card. The failing line is

cudaError_t error_id = cudaGetDeviceCount(&deviceCount);

Whereas the successful one is

checkCudaErrors(cuDeviceGetCount(&deviceCount));

The difference seems to be that the one from the driver API works and the one from the runtime API gives this strange return of 3. This is how the documentation describes the difference:

The driver and runtime APIs are very similar and can for the most part be used interchangeably. However, there are some key differences worth noting between the two.

Complexity vs. control

The runtime API eases device code management by providing implicit initialization, context management, and module management. This leads to simpler code, but it also lacks the level of control that the driver API has.

In comparison, the driver API offers more fine-grained control, especially over contexts and module loading

I’m not sure what is actually happening here and would rather use the runtime API, but at least there is a way to make it work. I suspect that some implicit thing or initialisation is not happening before the runtime API call and that this must be a bug if nVidia’s own sample code isn’t working!

3

I fixed this by upgrading to a Quadro P400. I think your issue will be because you are still on the K architecture which the runtime API seems to be incompatible with?