Hi,
I’m getting the below error when running the CUDA container using below command:nvcr.io/nvidia/k8s/cuda-sample:nbody nbody -gpu -benchmark
docker: Error response from daemon: OCI runtime create failed: container_linux.go:367: starting container process caused: process_linux.go:495: container init caused: Running hook #0:: error running hook: exit status 1, stdout: , stderr: nvidia-container-cli: initialization error: driver error: failed to process request: unknown.
As advised CUDA on WSL :: CUDA Toolkit Documentation , I’m raising the request to guide me how can this issue be resolved. Please let me know what information do you need to help me resolving this issue.
Providing some more info:
$ inxi -G
Regards
こんばんは。
私も同様のエラーが発生しました。
以下の記事に該当していませんか?https://github.com/microsoft/WSL/issues/6773
It’s a known issue on WIP 21354 that is semi-fixed on 21359.
Even then, you have to stop WSL from doing a driver version check by using a NVIDIA_DISABLE_REQUIRE=1 environment variable, as per ch14ota’s link below. E.g.:
docker run --gpus all --env NVIDIA_DISABLE_REQUIRE=1 nvcr.io/nvidia/k8s/cuda-sample:nbody nbody -gpu -benchmark
Good luck!
4 Likes
artem5
April 15, 2021, 10:35pm
4
I also cannot get wsl to work with my GPU. Very annoying…
Build: 21359.1
(running export NVIDIA_DISABLE_REQUIRE=1)
root@Ubuntu20:/# cat /etc/os-release | grep VERSION
VERSION="20.04 LTS (Focal Fossa)"
VERSION_ID="20.04"
VERSION_CODENAME=focal
root@DESKTOP-F5AMOPO:/# cat /proc/version
Linux version 5.4.91-microsoft-standard-WSL2 (oe-user@oe-host) (gcc version 9.3.0 (GCC)) #1 SMP Mon Jan 25 18:39:31 UTC 2021
root@Ubuntu20:/# inxi -G
Graphics: Device-1: Microsoft driver: dxgkrnl v: N/A
Device-2: Microsoft driver: dxgkrnl v: N/A
Display: server: No display server data found. Headless machine? tty: 120x30
Message: Advanced graphics data unavailable in console for root.
root@DESKTOP-F5AMOPO:/# nvidia-smi.exe # <--- native nvidia-smi
Thu Apr 15 23:29:42 2021
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 470.14 Driver Version: 470.14 CUDA Version: 11.3 |
|-------------------------------+----------------------+----------------------+
| GPU Name TCC/WDDM | Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|===============================+======================+======================|
| 0 NVIDIA GeForce ... WDDM | 00000000:2D:00.0 On | N/A |
| 0% 45C P8 18W / 350W | 580MiB / 24576MiB | 0% Default |
| | | N/A |
+-------------------------------+----------------------+----------------------+
| 1 NVIDIA GeForce ... WDDM | 00000000:2E:00.0 On | N/A |
| 0% 43C P8 10W / 200W | 280MiB / 8192MiB | 0% Default |
| | | N/A |
+-------------------------------+----------------------+----------------------+
+-----------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=============================================================================|
| 0 N/A N/A 1708 C+G Insufficient Permissions N/A |
| 0 N/A N/A 7820 C+G C:\Windows\explorer.exe N/A |
| 0 N/A N/A 8836 C+G ...5n1h2txyewy\SearchApp.exe N/A |
| 0 N/A N/A 9504 C+G ...IA GeForce Experience.exe N/A |
| 0 N/A N/A 10716 C+G ...lPanel\SystemSettings.exe N/A |
| 0 N/A N/A 11964 C+G ...2txyewy\TextInputHost.exe N/A |
| 0 N/A N/A 12836 C+G ...5n1h2txyewy\SearchApp.exe N/A |
| 0 N/A N/A 13776 C+G ...y\ShellExperienceHost.exe N/A |
| 0 N/A N/A 15100 C+G ...8bbwe\WindowsTerminal.exe N/A |
| 1 N/A N/A 1708 C+G Insufficient Permissions N/A |
+-----------------------------------------------------------------------------+
root@Ubuntu20:/# nvidia-smi # <--- the wsl one
Failed to initialize NVML: GPU access blocked by the operating system
Failed to properly shut down NVML: GPU access blocked by the operating system
root@Ubuntu20:/# /usr/local/cuda/samples/1_Utilities/deviceQuery/deviceQuery
/usr/local/cuda/samples/1_Utilities/deviceQuery/deviceQuery Starting...
CUDA Device Query (Runtime API) version (CUDART static linking)
cudaGetDeviceCount returned 35
-> CUDA driver version is insufficient for CUDA runtime version
Result = FAIL
root@Ubuntu20:/# ls /usr/lib/wsl/lib/ -la
total 54408
dr-xr-xr-x 1 root root 4096 Apr 15 22:18 .
drwxr-xr-x 4 root root 4096 Apr 14 23:47 ..
-r-xr-xr-x 1 root root 137264 Mar 18 03:41 libcuda.so
-r-xr-xr-x 1 root root 137264 Mar 18 03:41 libcuda.so.1
-r-xr-xr-x 1 root root 137264 Mar 18 03:41 libcuda.so.1.1
-r-xr-xr-x 1 root root 6053064 Mar 18 03:41 libnvcuvid.so.1
-r-xr-xr-x 1 root root 424440 Mar 18 03:41 libnvidia-encode.so.1
-r-xr-xr-x 1 root root 197104 Mar 18 03:41 libnvidia-ml.so.1
-r-xr-xr-x 1 root root 354808 Mar 18 03:41 libnvidia-opticalflow.so.1
-r-xr-xr-x 1 root root 47582360 Mar 18 03:41 libnvwgf2umx.so
-r-xr-xr-x 1 root root 670104 Mar 18 03:41 nvidia-smi
Unfortunately this is still not fixed.
None of the fixes described above work!
:-( :-( :-(
4 Likes
Was trying to run jetson deepstream docker image on x86 machine and was getting the above error --env NVIDIA_DISABLE_REQUIRE=1 helped me, thanks @damianlewis