Nvidia-smi through WSL2

Here is my PC build.

OS: Windows 10 Pro, 64-Bit
GPU: NVIDIA GeForce GTX 1080 Ti x 4
CPU: AMD Ryzen Threadripper 1950X 16-Core Processor

I was able to successfully install WSL2 and Ubuntu 20.04. I also installed Windows Terminal to run the commands.

I tried installing CUDA and it seems that did not actually work.

  1. I downloaded the WSL drivers from: https://developer.nvidia.com/cuda/wsl/download

I downloaded and ran the exe: 470.76_gameready_win10-dch_64bit_international.exe

  1. I then following this user guide: CUDA on WSL :: CUDA Toolkit Documentation

I ran:

  1. Step 2 seemed to have run successfully but when I try running nvidia-smi I get:

This makes me think the CUDA installation did not actually work.

what’s the the Windows build you are running? (winver.exe )

Microsoft Windows
Version 20H2 (OS Build 19042.1052)

As stated here, CUDA in WSL2 is only available in build 20145 and superior. For now is only for Insider builds in the Dev channel but it should be available for everyone in October-November in the upcoming Windows 10 version 21H2.

Thanks for the answer. So what is the best way to “fix” my installation since I have already installed the driver and wsl2? Simply enable the Insider builds, or do I have to uninstall prior items first?

Just enable the Dev channel for Insider builds and after a while the build will appear on Windows Update.

One more question: will this allow me to do multiple GPU training in PyTorch using DDP? Does CUDA in WSL2 allow for this?

Since I don’t have multi-GPU I can’t really say. The nvidia driver is still in preview but they improved it a lot since the first release so maybe it’s already ready for that. Just make sure every gpu has a WDDM3.0 driver installed in Windows and they should appear inside WSL2.

1 Like

I’m on Win10 22H2 and everything seems to work(tensorflow,cuda examples) except there is no nvidia-smi, when I try to run it it showed similar output as the original post here. I don’t want to install the wrong one but how do I know which packages should be installed to get nvidia-smi:

Command 'nvidia-smi' not found, but can be installed with:

sudo apt install nvidia-340               # version 340.108-0ubuntu5.20.04.2, or
sudo apt install nvidia-utils-390         # version 390.144-0ubuntu0.20.04.1
sudo apt install nvidia-utils-450-server  # version 450.172.01-0ubuntu0.20.04.1
sudo apt install nvidia-utils-470         # version 470.103.01-0ubuntu0.20.04.1
sudo apt install nvidia-utils-470-server  # version 470.103.01-0ubuntu0.20.04.1
sudo apt install nvidia-utils-510         # version 510.54-0ubuntu0.20.04.1
sudo apt install nvidia-utils-510-server  # version 510.47.03-0ubuntu0.20.04.1
sudo apt install nvidia-utils-435         # version 435.21-0ubuntu7
sudo apt install nvidia-utils-440         # version 440.82+really.440.64-0ubuntu6
sudo apt install nvidia-utils-418-server  # version 418.226.00-0ubuntu0.20.04.2

On WSL2 if everything is setup correctly the binary should be in your PATH and it is located in /usr/lib/wsl/lib. You should see nvidia-smi there.

If it is not there I would suggest double checking your driver version and trying installing a more recent driver: Official Drivers | NVIDIA

If it is there then I would double check that /usr/lib/wsl/lib is in your PATH. This is usually automatic but depending on your setup that might be overwritten or configured differently.

Thanks !

2 Likes

Thank you! it’s there. That directory is fully of libraries, why an executable ended up there is interesting.

This is because that path is the path that gets mounted (from C:/System32/lxss) in order to provide some of the loader libraries and core library for WSL. Since nvidia-smi is usually provided with our driver its copy gets put there so users can access it and don’t have to fish it from the DriverStore.

Now that path is usually added to your PATH by default as to make the call to nvidia-smi transparent. We are looking at more ways to safeguard this and make it even more transparent for WSL users but today this is where that executable is located.