Nsys CLI error in WSL2：No GPU associated to the given UUID

Developer Tools Nsight Systems Profiling Linux Targets
1

Hi, I try to use nsys CLI on the WSL2(ubuntu 22.04 + cuda 11.8 + host driver 536.25)，but when i try to profile a python file containing hugging face transformer calling cuda.

Errors come below:

(transformer) ubuntu@D3211230-IGU:/tmp/fdy$ nsys profile -w true -t cuda,nvtx,osrt,cudnn,cublas -s cpu  --cudabacktrace=true -x true -o my_profile python ./test.py 

<logs_in_test.py>

Generating '/tmp/nsys-report-d595.qdstrm'
FATAL ERROR: /build/agent/work/323cb361ab84164c/QuadD/Common/GpuTraits/Src/GpuTicksConverter.cpp(376): Throw in function QuadDCommon::TimestampType GpuTraits::GpuTicksConverter::ConvertToCpuTime(const QuadDCommon::Uuid&, uint64_t&) const
Dynamic exception type: boost::wrapexcept<QuadDCommon::NotFoundException>
std::exception::what: NotFoundException
[QuadDCommon::tag_message*] = No GPU associated to the given UUID

and my nsys version in WSL2 is:

(transformer) ubuntu@D3211230-IGU:/tmp/fdy$ nsys -v
NVIDIA Nsight Systems version 2022.4.2.1-df9881f

Could you help me? how to use nsys correctly in WSL2 or it’s unsupported?

2

@jasoncohen, could you please respond to this.

3

I download the latest version of Nsight Systems manually:

NVIDIA Nsight Systems version 2023.2.1.122-32598524v0

then it works. BUT it’s still a unresonable mistake that I installed Nsight before through the CUDA runfile by following the download command below:

wget https://developer.download.nvidia.com/compute/cuda/11.8.0/local_installers/cuda_11.8.0_520.61.05_linux.run
sudo sh cuda_11.8.0_520.61.05_linux.run

the shipped Nsight in runfile is 2022.4.2.1-df9881f，and it DOESN’T work.
Holp you guys can fix this bug. appreciate!

4

@hwilper I fix this bug by downloading the latest version of Nsight manually.
however it should be a bug in WSL2 CUDA runfile which shipped the unsupported Nsight version.
Please check these potential mistakes, maybe wrong on some other platforms or versions.
Thanks for your reply

5

Will do, and thank you for reporting back with this.