Understanding CUDA architecture when using WSL

I am relatively new to using CUDA so I have a hard time understanding its architecture. So I have few questions.

  1. Is there a better documentation than CUDA on WSL User Guide?
  2. In the above picture, which step of CUDA on WSL User Guide installs what on which layer?
    a) Is (A) in above picture, installed by following Installing NVIDIA DriversCUDA on WSL: CUDA on Windows Subsystem for Linux (WSL) - Public PreviewGet CUDA Driver: NVIDIA Drivers for CUDA on WSL, including DirectML Support → 470.76_gameready_win10-dch_64bit_international.exe?
            i) If the answer to above in a) is yes, then how is it different from correctly downloading and installing from NVIDIA Driver Downloads? For my case, it will be GEFORCE GAME READY DRIVER (466.63 WHQL) {2021.6.3}? The version seems to be different from a) (470.76_gameready_win10-dch_64bit_international.exe) but what is the difference?
    b) Is (B) in above picture, installed by following Setting up CUDA Toolkit? If not where does CUDA Toolkit fit in, in the above picture? Is the toolkit necessary at all?
    c) Although CUDA on WSL User Guide doesn’t mention cuDNN, it seems cuDNN’s version is very important for tensorflow build configuration. In tensorflow build configuration for GPU, as shown in picture below, what is CUDA header column referring to, do they mean the version of CUDA Toolkit?

  1. Assuming installation of CUDA Toolkit is necessary, there seems to be 2 ways for WSL as shown in bullet points below. However the execution command seems to be different. So is it really 2 ways or do I need to install both in succession?

$ apt-key adv --fetch-keys http://developer.download.nvidia.com/compute/cuda/repos/ubuntu2004/x86_64/7fa2af80.pub
$ sh -c ‘echo “deb http://developer.download.nvidia.com/compute/cuda/repos/ubuntu2004/x86_64 /” > /etc/apt/sources.list.d/cuda.list’
$ apt-get update
$ apt-get install -y cuda-toolkit-11-1

wget https://developer.download.nvidia.com/compute/cuda/repos/wsl-ubuntu/x86_64/cuda-wsl-ubuntu.pin
sudo mv cuda-wsl-ubuntu.pin /etc/apt/preferences.d/cuda-repository-pin-600
wget https://developer.download.nvidia.com/compute/cuda/11.1.0/local_installers/cuda-repo-wsl-ubuntu-11-1-local_11.1.0-1_amd64.deb
sudo dpkg -i cuda-repo-wsl-ubuntu-11-1-local_11.1.0-1_amd64.deb
sudo apt-key add /var/cuda-repo-wsl-ubuntu-11-1-local/7fa2af80.pub
sudo apt-get update
sudo apt-get -y install cuda

As a side note, the reason why I am installing CUDA Toolkit v11.1 is because in the end I would like to execute the following command for PyTorch (Stable: 1.8.1, Linux, Conda, Python, CUDA 11.1)

conda install pytorch torchvision torchaudio cudatoolkit=11.1 -c pytorch -c nvidia

Hello and thanks for your questions,
This is indeed a point that is often confusing and could use some clarification.

  • First let’s talk about the CUDA Driver (you need it to run CUDA programs):

Our classic Windows Display Driver always includes the CUDA Driver for native Windows.
When it comes to WSL, the CUDA Driver for WSL is also included in the same Windows Display Driver.
This means that the Windows Display Driver Comes with both the Native and the WSL CUDA Driver.

That said because WSL2 GPU support is still in preview, we only include the WSL Drivers as part of the Windows Display Driver in some of our builds. Namely, the drivers you get automatically if you are on the Windows Insider Program via Windows Update and the driver you can download manually on the CUDA Developer portal here.

When WSL2 GPU support will no longer be in preview, we plan to make WSL Drivers part of the Windows Display Driver all the time. At that point, yes, the driver you will get from the NVIDIA Driver Download webpage for instance, will come built in with WSL Drivers. Until then, only the specific drivers mentioned above have the WSL CUDA Driver built-in.

  • Now let’s talk about the CUDA Toolkit (you need it to develop CUDA Program):

Usually the toolkit is needed for you to develop CUDA Programs, not to run them. Most program developed with CUDA will redistribute themselves the components from the toolkit they need to run. The reason why we have separate packages for WSL is to safeguard users that might accidentally install not just the toolkit but the full package “install cuda” which by default will also install the native linux CUDA driver. The WSL variant is exactly the same as the regular one, just without the driver dependency :).

An other easy way to avoid any issues would be to download the .run file for the CUDA toolkit. Those will be able to check for WSL and skip the native driver install. We are constantly trying to find ways to avoid this, as we realize this is one of the biggest source of confusion and setup issues right now.

I hope this could answer your questions, and thanks a lot for trying WSL GPU support,

2 Likes

@rboissel Thank you for your response. I understand more now. After reading your response, here is my understanding towards my questions. Also parts in italic below are my own assumptions.

  • Answer to Question 1: Documentation

Since you did not mention a different site and needed to clarify, I assume there isn’t a better documentation on CUDA architecture than CUDA on WSL User Guide.

The reason why we have separate packages for WSL is to safeguard users that might accidentally install not just the toolkit but the full package “install cuda” which by default will also install the Native Linux CUDA Driver.

So this is to avoid accidentally overwriting NVIDIA Drivers for CUDA on WSL with NVIDIA Native Linux Driver by executing sudo apt-get install cuda.

An other easy way to avoid any issues would be to download the .run file for the CUDA toolkit. Those will be able to check for WSL and skip the native driver install.

I might be just rewording your response but … so I can still go through CUDA Toolkit 11.1.0 (archive) download site and avoid overwriting NVIDIA Drivers for CUDA on WSL by selecting the runfile (local) as shown below (this is regardless of whether NVIDIA Native Linux CUDA Driver (e.g. Ubuntu 20.04) is selected):

1 Like

You got all of it right :)

Thanks!

1 Like