Failed cuDNN test (./mnistCUDNN)

This seems to be a very general error, but maybe someone have got the same problem.

ubuntu@ubuntu:$ ./mnistCUDNN 
cudnnGetVersion() : 7003 , CUDNN_VERSION from cudnn.h : 7003 (7.0.3)
Cuda failurer version : GCC 5.4.0
Error: CUDA driver version is insufficient for CUDA runtime version
error_util.h:93
Aborting...

currently running:
CUDA 9.0
cudnn 7.0
NVIDIA Driver 384.90
Ubuntu 16.04

Any ideas how to proceed?

Had the same problem…

install the driver properly. Use the method in the linux install guide. Perform all steps including verification, before attempting to do anything else (like install cudnn)

Had the same problem …

cudnnGetVersion() : 7005 , CUDNN_VERSION from cudnn.h : 7005 (7.0.5)
Cuda failurer version : GCC 5.4.0
Error: CUDA driver version is insufficient for CUDA runtime version
error_util.h:93
Aborting…

Hi there,

I had the same problem running the examples (mnistCUDNN). I am getting the following error.
Any help would be appreciated.

cudnnGetVersion() : 7005 , CUDNN_VERSION from cudnn.h : 7005 (7.0.5)
Cuda failurer version : GCC 5.4.1
Error: unknown error
error_util.h:93
Aborting...

has same problem.

eh…I fix this problem by reboot my computer, and samples works fine now.

4 Likes

rebooting fixed it for me as well.

2 Likes

and me. arrrgh

$ sudo ./mnistCUDNN 
cudnnGetVersion() : 7005 , CUDNN_VERSION from cudnn.h : 7005 (7.0.5)
Cuda failurer version : GCC 5.4.0
Error: CUDA driver version is insufficient for CUDA runtime version
error_util.h:93
Aborting...

Rebooting does not work for me. I am using CUDA 9.0 with CuDnn 7.0.5.

restart worked…

However! Under System Settings → Software and Updates → Additional Drivers it looks as it changed the driver. I selected 390.48, but now it is set as 384.130, which is weird, as I think you need a 390.x driver due to cuda 9.0. Edit: Apparently it’s not a 100% requirement, just very recommended as newer Kernels will need the newer driver.

(ubuntu 16.04)

I disabled secure boot while rebooting and it worked for me.

Just wanted to comment that I had disabled secure boot and got this same error.

Restarting the computer, like others mentioned, made it work, getting the following from ./mnistCUDNN:
cudnnGetVersion() : 7301 , CUDNN_VERSION from cudnn.h : 7301 (7.3.1)
Host compiler version : GCC 5.4.0
There are 1 CUDA capable devices on your machine :
device 0 : sms 28 Capabilities 6.1, SmClock 1582.0 Mhz, MemSize (Mb) 11164, MemClock 5505.0 Mhz, Ecc=0, boardGroupID=0
Using device 0

Testing single precision
Loading image data/one_28x28.pgm
Performing forward propagation …
Testing cudnnGetConvolutionForwardAlgorithm …
Fastest algorithm is Algo 1
Testing cudnnFindConvolutionForwardAlgorithm …
^^^^ CUDNN_STATUS_SUCCESS for Algo 0: 0.021504 time requiring 0 memory
^^^^ CUDNN_STATUS_SUCCESS for Algo 1: 0.027648 time requiring 3464 memory
^^^^ CUDNN_STATUS_SUCCESS for Algo 2: 0.034816 time requiring 57600 memory
^^^^ CUDNN_STATUS_SUCCESS for Algo 7: 0.069632 time requiring 2057744 memory
^^^^ CUDNN_STATUS_SUCCESS for Algo 4: 0.070656 time requiring 207360 memory
Resulting weights from Softmax:
0.0000000 0.9999399 0.0000000 0.0000000 0.0000561 0.0000000 0.0000012 0.0000017 0.0000010 0.0000000
Loading image data/three_28x28.pgm
Performing forward propagation …
Resulting weights from Softmax:
0.0000000 0.0000000 0.0000000 0.9999288 0.0000000 0.0000711 0.0000000 0.0000000 0.0000000 0.0000000
Loading image data/five_28x28.pgm
Performing forward propagation …
Resulting weights from Softmax:
0.0000000 0.0000008 0.0000000 0.0000002 0.0000000 0.9999820 0.0000154 0.0000000 0.0000012 0.0000006

Result of classification: 1 3 5

Test passed!

Testing half precision (math in single precision)
Loading image data/one_28x28.pgm
Performing forward propagation …
Testing cudnnGetConvolutionForwardAlgorithm …
Fastest algorithm is Algo 1
Testing cudnnFindConvolutionForwardAlgorithm …
^^^^ CUDNN_STATUS_SUCCESS for Algo 0: 0.024480 time requiring 0 memory
^^^^ CUDNN_STATUS_SUCCESS for Algo 1: 0.027488 time requiring 3464 memory
^^^^ CUDNN_STATUS_SUCCESS for Algo 2: 0.047104 time requiring 28800 memory
^^^^ CUDNN_STATUS_SUCCESS for Algo 4: 0.068608 time requiring 207360 memory
^^^^ CUDNN_STATUS_SUCCESS for Algo 7: 0.069632 time requiring 2057744 memory
Resulting weights from Softmax:
0.0000001 1.0000000 0.0000001 0.0000000 0.0000563 0.0000001 0.0000012 0.0000017 0.0000010 0.0000001
Loading image data/three_28x28.pgm
Performing forward propagation …
Resulting weights from Softmax:
0.0000000 0.0000000 0.0000000 1.0000000 0.0000000 0.0000714 0.0000000 0.0000000 0.0000000 0.0000000
Loading image data/five_28x28.pgm
Performing forward propagation …
Resulting weights from Softmax:
0.0000000 0.0000008 0.0000000 0.0000002 0.0000000 1.0000000 0.0000154 0.0000000 0.0000012 0.0000006

Result of classification: 1 3 5

Test passed!

And I wanted to shout out to Robert_Crovella, wrt his instructions before, that I had done all of these:
“install the driver properly. Use the method in the linux install guide. Perform all steps including verification, before attempting to do anything else (like install cudnn)”

I came here after having done those. And indeed, the restart at the end of everything is what was needed. (That, and another pre-req is that secure boot is disabled.)

I also have the same problem, and reboot can work !

1 Like

Hi I have the same issue with

CUDA 9.0
cudnn 7.0
NVIDIA Driver 384.145 (but also tried 390)
Ubuntu 16.04
GPU 720M

Rebooting or using sudo for the cdnn test is not working

I still get this

cudnnGetVersion() : 7005 , CUDNN_VERSION from cudnn.h : 7005 (7.0.5)
Host compiler version : GCC 5.4.0
There are 1 CUDA capable devices on your machine :
device 0 : sms 2 Capabilities 2.1, SmClock 1250.0 Mhz, MemSize (Mb) 1985, MemClock 800.0 Mhz, Ecc=0, boardGroupID=0
Using device 0

Testing single precision
CUDNN failure
Error: CUDNN_STATUS_ARCH_MISMATCH
mnistCUDNN.cpp:394
Aborting…

Thanks a lot for your help

Hello, im dealing qith the same isuue, im using a Nvidia Quadro 2000 with ubuntu 16.04+CUDA9.0+cudnn7.5
Any help or advice would be appreciated

~/cudnn_samples_v7/mnistCUDNN$ ./mnistCUDNN
cudnnGetVersion() : 7500 , CUDNN_VERSION from cudnn.h : 7500 (7.5.0)
Host compiler version : GCC 5.4.0
There are 1 CUDA capable devices on your machine :
device 0 : sms 4 Capabilities 2.1, SmClock 1251.0 Mhz, MemSize (Mb) 962, MemClock 1304.0 Mhz, Ecc=0, boardGroupID=0
Using device 0

Testing single precision
CUDNN failure
Error: CUDNN_STATUS_ARCH_MISMATCH
mnistCUDNN.cpp:394
Aborting…

After some time hitting my head against my desk, i understand than in my particular case, Nvidia quaddro 2000 card does not support Cudnn. Beause of architecture stuff i dunno. This post in particular helped a lot

So i cheecked in wikipedia

So now i know than nvidia quaddro 2000 card belongs to Fermi microarchitecture, which supports 2.1 arch. But cudnn needs 3.0 or more to work at runtime. So i guess im stopping in my attempts to make this cudnn work. Anyway, i still manage to use opencv with cuda with my card and i guess that might be enough for now. If anyone had the same issue and manage to make cudnn work out I’d like to know how to do that.

Hello,

You are right, cuDNN doesn’t work with Fermi architecture which is having the compute capability of 2.1. It is clearly mentioned in the Nvidia installation guide that cuDNN requires the GPU of the compute capability 3.0 or higher.

I think it’s just a waste of time in searching for a solution.

Rebooting worked Thanks

I have the same problems as follows, even restart again.

cudnnGetVersion() : 7605 , CUDNN_VERSION from cudnn.h : 7605 (7.6.5)
Cuda failurer version : GCC 7.4.0
Error: no CUDA-capable device is detected
error_util.h:93

I used graphic card GTX 960M, cuda version 10.0, ubuntu 18.04. Rather than installing the cuda as mentioned in the linux installation guide, the cuda was installed through Jackson SDK because i try to work on Jackson Nano through Nsight eclipse . who can help to resolve! thanks a lot!

solved by reboot +1