Nvidia-smi is not working in WSL2 Ubuntu 22.04.1 installed on windows server 2022

I have new windows server where I have installed WSL2 for GPU pass through, but nvidia-smi command is not working and it gives NVML initialization : unknown error whereas nvidia-smi.exe is working from the windows side

However, It works on another server with similar setup

Failed to initialize NVML: Unknown Error
nvidia-bug-report.log (161.9 KB)

Please find the nvidia bug report log attached here.

Hello,

Welcome to the forums. The regional section does not get much attention, so I moved your post to the GPU Hardware category for better visibility.

Cheers,
Tom

1 Like

Hi @praveenkumar.krish,

how did you create this log output? WSL2 by default does not contain that script unless you have installed an NVIDIA driver inside the VM. Which you explicitly should not do.

Please refer to the CUDA on WSL instructions and check if you followed them closely.

Beside that, the log file does not contain any information on any NVIDIA GPU either.

If you are running in a server environment with Enterprise GPUs (and support) you might not be able to use WSL at all but should rather look at our VGPU services.

Thanks!

Thanks @MarkusHoHo for the response. I will give a try by following the document shared and let you know if I face any issues.

@MarkusHoHo

Could you please confirm the below GPUs are supported in WSL2 for GPU pass through

Nvidia T1000 8GB
Nvidia A100 80GB PCIe
nvidia-smi

First one says its disabled, how to enable to make it working with WSL2

If you check out the main difference in the picture, it is with respect to TCC vs WDDM. Refer to the section “Driver Model” in the nvidia-smi documentation.

A100 is a compute only GPU, so it will always be in TCC mode. But for GPU pass-thru to work with WSL it would need to be WDDM.

T1000 is an older card which has different driver and compute model support.

Thanks @MarkusHoHo for the details shared. I will probably go with Hyper-V approach as WSL2 has limited features for GPU pass through

1 Like

@MarkusHoHo , Can you please confirm if NVIDIA A100 in TCC mode supports on GPU pass through inside Hyper-V Ubuntu VM using DDA?

I have tried to allocate the GPU to Hyper-V Ubuntu VM and installed the driver inside the VM but nvidia-smi command still throwing the below error,

“NVIDIA-SMI has failed because it couldn’t communicate with the NVIDIA driver. Make sure that the latest NVIDIA driver is installed and running.”

Sorry for the delay.

Virtualization of Compute GPUs like the A100 requires our Virtual GPU support. I am sorry if that was unclear before.

Thank you!

Thanks @MarkusHoHo . This issue has been fixed in Hyper-V after assigning the required MMIO space on the VM,

image

1 Like

Oh, perfect, thanks for updating!

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.