Nsys doesn't show cuda kernel and memory data

Developer Tools Nsight Systems Profiling Linux Targets
,
9

In WSL2, Nsys still has some issue with GPU->CPU timestamp conversion, so the GPU side CUDA activities are not processed correctly and not show up on timeline by default.

There is a WAR to let CUPTI, the underlying library that Nsys uses for CUDA trace, to handle the timestamp conversion. It will not be as accurate as Nsys’ normal mechanism, but the GPU side activities can show up on timeline.

Firstly, you might need to download the latest 2024.7 release from Nsight Systems - Get Started | NVIDIA Developer. There’s a bug in the CUPTI we used in older versions that prevented the WAR from working.

After that, here’s the steps for the WAR:

  1. Find the Nsys config.ini file path from nsys -z. For example on my system:
$ nsys -z
/home/liuyis/.config/NVIDIA Corporation/nsys-config.ini
  1. Create the config.ini file if it does not already exist. Note the path might have a space in it so it needs to be wrapped by quotes
mkdir -p "/home/liuyis/.config/NVIDIA Corporation"
touch "/home/liuyis/.config/NVIDIA Corporation/nsys-config.ini"
  1. Add a line in the config file: CuptiUseRawGpuTimestamps=false
echo "CuptiUseRawGpuTimestamps=false" > "/home/liuyis/.config/NVIDIA Corporation/nsys-config.ini"

Let me know if this works, thanks

1 Like