I can’t really find anything about error 103. I used the .run installer for the cuda-toolkit-11. Currently nvidia-docker does work properly, and I’m able to run the sample containers. However, locally, I’m not able to use the CUDA Toolkit. Here is some information.
./deviceQuery Starting…
CUDA Device Query (Runtime API) version (CUDART static linking)
cudaGetDeviceCount returned 103
→ integrity checks failed
Result = FAIL
$ nvidia-smi
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 510.00 Driver Version: 510.06 CUDA Version: 11.6 |
|-------------------------------+----------------------+----------------------+
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|===============================+======================+======================|
| 0 Quadro RTX 5000 On | 00000000:01:00.0 Off | N/A |
| N/A 0C P3 32W / N/A | 214MiB / 16384MiB | N/A Default |
| | | N/A |
+-------------------------------+----------------------+----------------------+
+-----------------------------------------------------------------------------+
| Processes: | |
GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=============================================================================|
| No running processes found |
+-----------------------------------------------------------------------------+
$ nvcc --version
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2022 NVIDIA Corporation
Built on Tue_Mar__8_18:18:20_PST_2022
Cuda compilation tools, release 11.6, V11.6.124
Build cuda_11.6.r11.6/compiler.31057947_0
$ uname -r
5.10.102.1-microsoft-standard-WSL2
$ cat /proc/version
Linux version 5.10.102.1-microsoft-standard-WSL2 (oe-user@oe-host) (x86_64-msft-linux-gcc (GCC) 9.3.0, GNU ld (GNU Binutils) 2.34.0.20200220) #1 SMP Wed Mar 2 00:30:59 UTC 2022
$ ls -l /mnt/c/Windows/System32/lxss/lib/ total 66584
dr-xr-xr-x 1 pipemon pipemon 512 Dec 23 12:02 ./
dr-xr-xr-x 1 pipemon pipemon 512 Mar 15 07:00 …/
-r-xr-xr-x 1 pipemon pipemon 141464 Aug 24 2021 libcuda.so*
-r-xr-xr-x 1 pipemon pipemon 141464 Aug 24 2021 libcuda.so.1*
-r-xr-xr-x 1 pipemon pipemon 141464 Aug 24 2021 libcuda.so.1.1*
-r-xr-xr-x 2 pipemon pipemon 828840 Sep 16 2021 libd3d12.so*
-r-xr-xr-x 2 pipemon pipemon 4834848 Sep 16 2021 libd3d12core.so*
-r-xr-xr-x 2 pipemon pipemon 878768 Sep 16 2021 libdxcore.so*
-r-xr-xr-x 1 pipemon pipemon 5962912 Aug 24 2021 libnvcuvid.so.1*
-r-xr-xr-x 1 pipemon pipemon 424400 Aug 24 2021 libnvidia-encode.so.1*
-r-xr-xr-x 1 pipemon pipemon 197752 Aug 24 2021 libnvidia-ml.so.1*
-r-xr-xr-x 1 pipemon pipemon 354768 Aug 24 2021 libnvidia-opticalflow.so.1*
-r-xr-xr-x 1 pipemon pipemon 53568104 Aug 24 2021 libnvwgf2umx.so*
-r-xr-xr-x 1 pipemon pipemon 682552 Aug 24 2021 nvidia-smi*
$ echo $LD_LIBRARY_PATH
/usr/local/cuda-11.6/lib64:/usr/local/lib:/usr/local/cuda-11.6/lib64:/usr/local/lib:
$ echo $PATH
contains… /usr/local/cuda-11.6/bin: