CUDA sample throwing error

A solution that worked for me Try:

  1. Opening a Windows Start Menu
  2. Type “Run” then enter
  3. Type “control /name Microsoft.DeviceManager” then enter
  4. Under “Device adapters” verify there’s no warnings or issues with Nvidia device
  5. If Windows states there’s a problem with device, follow simplified instructions provided by Microsoft at resource: Fix graphics device problems with error code 43
1 Like

I had the same problem and was banging my head for half a day. My problem was that I installed the WSL compatible driver (https://developer.nvidia.com/cuda/wsl/download ) before switching to new Windows Build which supports CUDA on WSL 2. I found for instance that libcuda.so was missing from C:\Windows\System32\lxss\lib. I had to re-install the wsl driver and after re-installing the driver everything worked.

2 Likes

I’m seeing the same problem and I’ve been through this entire post and tried everything. Here’s what I have done:

I have an Alienware M17 with GeForce RTX 2080 Super
Downloaded NVIDIA drivers for WSL (https://developer.nvidia.com/cuda/wsl/download)
Subscribed to Windows Insider Dev and applied the updates
Activated WSL 2 to give me Linux version 4.19.128-microsoft-standard
Verified that I have /usr/lib/wsl (to prove i have insider dev active)
Dowloaded CUDA 11.1 (Linux WSL-Ubuntu 2.0 x86_64) via CUDA Toolkit 11.7 Update 1 Downloads | NVIDIA Developer using deb(network) version.
Installed Cuda 11.1 via the recommended steps on that page
Checked and I do have files in /usr/local/cuda-11.1 to compile against.
Ensured that the Makefile in /usr/local/cuda-11.1/samples/0_Simple/clock use ‘CUDA_PATH ?= /usr/local/cuda-11.1’
Checked that the files /usr/lib/wsl/lib/libcuda.so.1 etc. exist
Ran strace on the ‘clock’ program and verified that it is loading ‘/usr/lib/wsl/lib/libcuda.so.1’
After all this I still get the error ‘CUDA error at …/…/common/inc/helper_cuda.h:779 code=35(cudaErrorInsufficientDriver) “cudaGetDeviceCount(&device_count)”’

I’m running out of ideas. Can anyone please help?

2 Likes

Hello everyone. I’m facing the same problem, but despite having followed all the guidelines in the post, nothing went right. Here is my procedure:

I have a Dell Inspiron 15 7000 Gaming notebook, with an NVIDIA GeForce GTX 1050 Ti card, and Windows 10 Pro Insider Preview, 2004, 21277.1000, with Windows Update Advanced options enabled to enable recommended Microsoft updates and run the check again . Several times I followed the tutorial step ‘CUDA on WSL :: CUDA Toolkit Documentation’ for installing and configuring the CUDA Toolkit:


Installing NVIDIA Drivers on Windows:
460.89 - 12/15/2020


Installing WSL 2: I had already installed it in advance through the tutorial ‘Instalar o WSL | Microsoft Docs’ - 9/15/2020


PowerShell (admin):

Linux version:
PS C:\Users\sanam> wsl cat /proc/version
Linux version 5.4.72-microsoft-standard-WSL2 (oe-user @ oe-host) (gcc version 8.2.0 (GCC)) # 1 SMP Wed Oct 28 23:40:43 UTC 2020

Ubuntu-18.04 installed by Microsoft Store:
PS C:\Users\sanam> wsl --list --verbose
NAME STATE VERSION
Ubuntu-18.04 Running 2

GPU recognition:
PS C:\Users\sanam> nvidia-smi

Activating Ubuntu:
PS C:\Users\sanam> wsl


Ubuntu CLI:

/mnt/c/Users/sanam$ cd ~
~$ sudo su

~# sudo apt update
~# sudo apt upgrade

Verify You Have a CUDA-Capable GPU (‘Installation Guide Linux :: CUDA Toolkit Documentation’):
~# update-pciids
Downloaded daily snapshot dated 2020-12-13 03:15:01
~# lspci | grep -i nvidia
“Nothing happens!”

Verify You Have a Supported Version of Linux:
~# uname -m
x86_64
~# cat /etc/*release

PRETTY_NAME = “Ubuntu 18.04.5 LTS”

Verify the System Has gcc Installed:
~# gcc --version
Command ‘gcc’ not found, but can be installed with:
apt install gcc
~# apt update
~# gcc --version
Copyright (C) 2017 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSEgcc (Ubuntu 7.5.0-3ubuntu1 ~ 18.04) 7.5.0

Verify the System has the Correct Kernel Headers and Development Packages Installed:
~# uname -r
5.4.72-microsoft-standard-WSL2

Setting up CUDA Toolkit:
~# apt-key adv --fetch-keys http://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64/7fa2af80.pub
~# sh -c ‘echo “deb Index of /compute/cuda/repos/ubuntu1804/x86_64 /”> /etc/apt/sources.list.d/cuda.list’
~# apt-get update
~# apt-get install -y cuda-toolkit-11-0

Running CUDA Applications:
~# cd /usr/local/cuda/samples/4_Finance/BlackScholes
~# make
~# ./BlackScholes
[./BlackScholes] - Starting …
CUDA error at …/…/common/inc/helper_cuda.h:777 code = 35 (cudaErrorInsufficientDriver) “cudaGetDeviceCount (& device_count)”

Since then I am looking for how to solve this problem. If possible, I would appreciate your help.

Thanks.

4 Likes

I encountered the exact same problem after update to CUDA 11.2 and 465.21 driver, I have only one WSL2 setup. I think it’s a NV bug. please help

4 Likes

Yeah dude I cannot fathom this shit dude , stuck since Sep-Nov

I’ve been doing in Beta channel. Thanks for the solution. I’ll try it again!

same problem here. I tried everything as said in manual, but nothing works out. NVIDIA team please solve.

Board: Jetson AGX XAVIER
Jetpack: 4.4.1

Dear, this time everything went well!
After the new update of Windows 10 Insider Preview 21286.1000 (rs_prerelease) and as well as the NVIDA driver to 465.21, I decided to try again, performing the following steps:
Note: You must already be enrolled in the Windows Insider Program.

  1. Uninstall all linux distributions.
  2. Uninstall WSL (right-click on the windows start logo> App and Features> Enable or disable windows features> uncheck the box for Windows Subsystem for Linux).
  3. Restart the computer.
  4. Open PowerShell in administrator mode and perform the simplified installation of WSL, as it already (i) enables the optional WSL and Virtual Machine Platform components, (ii) downloads and installs the latest Linux kernel, (iii) defines WSL 2 as the default and (iv) download and install an Ubuntu distribution (Just to inform you, I had previously performed the manual installation):

PS C:\Users\sanam> wsl.exe --install
PS C:\Users\sanam> exit

  1. Restart your computer, and wait while Ubuntu boots.
  2. Within Ubuntu, enter the new user account and password.
  3. Run the commands for installing and configuring the CUDA Toolkit:

$ sudo su

Enter your password to have root access and update Ubuntu.

/home/san# apt update
/home/san# apt upgrade

  1. Then, follow the CUDA Toolkit Documentation > CUDA on WSL tutorial at 1. NVIDIA GPU Accelerated Computing on WSL 2 — CUDA on WSL 12.3 documentation, which is the same as the video "GPU Accelerated Machine Learning with WSL 2 "from Youtube:

/home/san# apt-key adv --fetch-keys http://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64/7fa2af80.pub

/home/san# sh -c ‘echo “deb Index of /compute/cuda/repos/ubuntu1804/x86_64 /”> /etc/apt/sources.list.d/cuda.list’

/home/san# apt-get update

/home/san# apt-get install -y cuda-toolkit-11-0

  1. Running a CUDA application:

/home/san# cd /usr/local /cuda/samples/4_Finance/BlackScholes

/usr/local /cuda/samples/4_Finance/BlackScholes# make

/usr/local /cuda/samples/4_Finance/BlackScholes# ./BlackScholes

Ok. All right !!!
Thank you all!
Obrigado a todos!

3 Likes

I installed cuda-toolkit-11-2 from the ubuntu2004 repo, and am getting the error you previously had. Maybe the older version is the one required (it’s also what the guide instructs us to install, as you followed)

Uninstalled cuda-toolkit-11-2 and replaced with the older version, cuda-toolkit-11-0, which fixed the problem. Also I don’t think there’s a difference between the 1804 and 2004 repos, at least in regards to the toolkit and its dependencies

Upgrading to cuda-toolkit-11-2 breaks it for me. Downgrading to cuda-toolkit-11-0 fixes it. There’s some sort of change that breaks WSL support in the newer version.

Just pulling info from the other thread on the devicequery sample, I was mucking around with it and there’s something that’s not working in the latest 11-2, but i’m running 11-1 without issue (posted this over on https://forums.developer.nvidia.com/t/devicequery-sample-fails-to-run/163871/8?u=anthony.tan)

After 2-3 days of trouble shooting, this finally worked for me! Specifically using apt-get install -y cuda-toolkit-11-0 instead of $ apt-get install -y cuda. This was confusing, because the instructions here which specifically target WSL, says only cuda, but this is wrong.

I first had to uninstall cuda-11.2, using the instructions found here. I am on WSL, but used the instructions for Ubuntu (then finally manually deleted the /usr/local/cuda-11.2 directory).

I had also forgot to add the following lines to my .bashrc:
export PATH="/usr/local/cuda-11.0/bin:$PATH"
export LD_LIBRARY_PATH="/usr/local/cuda-11.0/lib64:$LD_LIBRARY_PATH"

Ah, yeah, there’s that. If you install the CUDA toolkit 11-2, it’ll install the samples to /usr/local/cuda-11.2 and symlink over /usr/local/cuda to that directory (via /etc/alternatives).

Depending on what you’re doing, instead of PATH patching, you might find it easier to grab the a Makefiles from the samples area for 11-1 and using that - you should see at the top the toolkit is defined with a conditional, so you could do:

make CUDA_PATH=/usr/local/specific/location/of/toolkit

(I haven’t done much playing with this other than getting the samples to work/run so can’t comment on the LD_LIBRARY_PATH thing though)

I did somewhat a similar process. Reinstalled the Linux distro and reinstall. I found in my case I had to reinstall the Nvidia driver again. Until I did that - kept getting the error. No clue why.

1 Like

I solved this problem by making sure my Windows Insider Program was set to the Dev Channel. It’s not super clear in the Nvidia installation guide that this is required. Thanks all for the suggestions and help.

Other notes:
As of March 2, 2021 - cuda toolkit 11.2 worked with BlackScholes:
apt-get install -y cuda-toolkit-11-2

I had problems with the benchmark Cuda container but the the TensorFlow Jupyter notebook ran fine

docker run -it --gpus all -p 8888:8888 tensorflow/tensorflow:latest-gpu-py3-jupyter

Good luck!

Reinstalling the Windows host’s graphics driver did the trick for me as well.
EDIT: One restart of the host later, it doesn’t work just like before.

Awesome, thanks, this worked for me. Realized I did the same (installed driver before the windows insiders build), so re-installed the driver and it works.

This worked for me. Thanks!