CUDA driver version is insufficient for CUDA runtime version

Accelerated Computing CUDA CUDA Setup and Installation
1

I am trying to run a CUDA code in a container hosted on a linux system. I have installed nvidia driver 520.61.5 and cuda runtime 11.8 as suggested in the release notes - Release Notes :: CUDA Toolkit Documentation. But still while running the cuda code I get the error “CUDA driver version is insufficient for CUDA runtime version”.

1 Like
2

The driver components in the container may be broken/mismatched. This can happen if you are not using the nvidia container toolkit. How did you launch the container?

3

This is how I run the container- sudo docker run --rm -it --runtime=nvidia wine/lungcad:v1

Also nvidia-smi lists the correct version nos

4

I expect it is a problem with container usage, of some sort.

You may not have an up-to-date nvidia container toolkit.

What happens if you run your code outside the container?

5

hi Robert,
I am colleague of Vivek.
We are getting the same error even outside the container
“CUDA error: CUDA driver version is insufficient for CUDA runtime version”
and i have installed nvidia container toolkit recently and i hope it is latest.

Below is the output of nvidia-smi:

±----------------------------------------------------------------------------+
| NVIDIA-SMI 520.61.05 Driver Version: 520.61.05 CUDA Version: 11.8 |
|-------------------------------±---------------------±---------------------+
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|===============================+======================+======================|
| 0 Quadro P1000 On | 00000000:0F:00.0 On | N/A |
| 34% 36C P8 N/A / N/A | 261MiB / 4096MiB | 1% Default |
| | | N/A |
±------------------------------±---------------------±---------------------+

±----------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=============================================================================|
| 0 N/A N/A 966 G /usr/lib/xorg/Xorg 39MiB |
| 0 N/A N/A 1610 G /usr/lib/xorg/Xorg 125MiB |
| 0 N/A N/A 1746 G /usr/bin/gnome-shell 13MiB |
| 0 N/A N/A 2297 G /usr/lib/firefox/firefox 12MiB |
| 0 N/A N/A 352509 G /usr/lib/xorg/Xorg 60MiB |
±----------------------------------------------------------------------------+

6

If you are running typical CUDA codes (e.g. vectorAdd sample code) on a CUDA 11.8 install, and getting this message, then your driver install is broken. You may have mixed driver components installed on your machine, depending on machine history/install history.

I won’t be able to diagnose it for you. You may wish to study the CUDA linux install guide. Also there are many forum posts that have troubleshooting suggestions. If in doubt, reload the operating system to remove all previous activity, and install CUDA 11.8 again. That method will always work, without having to unravel the complete history of your machine.

7

@Robert_Crovella I have tried reloading the OS, but the issue is not solved.
Actually, the application that I am trying to run is a Windows application for which I am using Wine. Any Wine specific configuration needs to be done to run the application with GPU?