Nsight Compute Fails To Profile Kernels on WSL Windows11

Developer Tools Nsight Compute
1

I have done a fresh installation of WSL and the CUDA toolkit on a Windows 11 machine.

My RTX 3080 is visible within the WSL and I can execute CUDA programs without any errors from the compute-sanitizer.
I am working with a toy cuda kernel which I want to profile.
When I run Nsight Compute, it fails to profile the kernel.
I also tried with a trivial kernel that only includes a “hello world” print, and I had the same results.
Executing in sudo mode gave the same error.

 ncu ./scratch
init C:1.73696  1.96054 0.317545        1.98527 1.43158 0.599105        1.63089 0.388541        0.0194331     0.597726
==PROF== Connected to process 27948 (/home/isakioti/scratch/study/build/scratch)


==ERROR== Failed to prepare kernel for profiling

==ERROR== Unknown Error on device 0.
==ERROR== Failed to profile "coalescing" in process 27948
==PROF== Trying to shutdown target application
==ERROR== The application returned an error code (9).
==ERROR== An error occurred while trying to profile.
==WARNING== No kernels were profiled.

Any ideas on how I can go about fixing this?
From my understanding, nothing in my system should cause any incompatibility issues, and I do have the latest drivers installed.

I am at a loss on what to check since I can execute the program fine and even Nsight Systems can profile it and generate a .nsys-rep file.

Below are some version info.

 nvidia-smi
Sat Mar 30 13:09:55 2024
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 550.65                 Driver Version: 551.86         CUDA Version: 12.4     |
|-----------------------------------------+------------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
|                                         |                        |               MIG M. |
|=========================================+========================+======================|
|   0  NVIDIA GeForce RTX 3080        On  |   00000000:01:00.0  On |                  N/A |
| 53%   42C    P0            114W /  370W |    1441MiB /  10240MiB |      1%      Default |
|                                         |                        |                  N/A |
+-----------------------------------------+------------------------+----------------------+

+-----------------------------------------------------------------------------------------+
| Processes:                                                                              |
|  GPU   GI   CI        PID   Type   Process name                              GPU Memory |
|        ID   ID                                                               Usage      |
|=========================================================================================|
|  No running processes found                                                             |
+-----------------------------------------------------------------------------------------+

 ncu --version
NVIDIA (R) Nsight Compute Command Line Profiler
Copyright (c) 2018-2023 NVIDIA Corporation
Version 2024.1.0.0 (build 33681293) (public-release)

nvcc --version
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2024 NVIDIA Corporation
Built on Tue_Feb_27_16:19:38_PST_2024
Cuda compilation tools, release 12.4, V12.4.99
Build cuda_12.4.r12.4/compiler.33961263_0
2

Hi, @iesak

Thanks for using Nsight Compute and sorry for the issue you met. Please provide us more details about this issue.

  1. Can you rebuild your sample and make sure “-gencode arch=compute_86,code=sm_86” is used ?
  2. Can Nsight Compute work properly on Windows ?
  3. Which driver version do you use ? I see 550.65 and 551.86 from the output.
3

Hi veraj. I was shocked to see that after a reboot, profiling worked. I thought I had tried but it seems that was not the case. I’m still having issues the performance but I think I may need to make a separate post for that after some further investigation.

4

Glad to hear about it ! Thanks for sharing this info !