Hello, I’m profiling my cuda program with Nsight System 2023.4.4.
But when i got nsys-rep, i got error message “CUDA device 0: Unified Memory cannot be traced on devices that don’t support peer-to-peer transfers.Please verify that SLI/NVLink is functioning properly.” and “CUDA device 1: Unified Memory cannot be traced on devices that don’t support peer-to-peer transfers.Please verify that SLI/NVLink is functioning properly.”.
It says that my devices don’t support peer-to-peer transfer, so it cannot trace any more.
I think i’m already check all the requirements.
I use 2 NVIDIA GeForce RTX 3090 devices which are both 552.22 driver version, core i9 cpu, windows 10.
Also my cuda program is version 11.7.
When i check the peer-to-peer memory accessability by “cudaDeviceCanAccessPeer”, it return 1 which means my 3090 gpus can support p2p memory access.
Also i enable peer access by “cudaDeviceEnablePeerAccess”.
The nsys-rep shows many things but error messages above appear, so i cannot believe the results of nsys-rep.
Can anyone help me? Thanks a lot.
1 Like
That’s a pretty old CUDA and Nsys with a pretty new driver, which shouldn’t be a problem, but might be worth updating your Nsys to 2024.4.
The only bad data you would have from this warning is that the UVM trace line would not be present and if you asked for UVM gpu or CPU page faults you would not get them. The rest of the data should be solid.
Hi hwilper,
i have met the same problem, but my CUDA and Nsys version is new.
CUDA: Cuda compilation tools, release 12.8, V12.8.61
Nsys: 12.8.0-1
And I developed and used Nsys on WSL2.
Can you help me?Thanks
There is a known issue with WSL that I think you are hitting, here is the note from the docs:
The default time conversion used by Nsight Systems is not reliable on WSL, and we need to fall back to a safer, but less precise time system. To do that please set the config file option CuptiUseRawGpuTimestamps to false. The config file option can be set using the following commands:
mkdir -p “$(dirname “$(nsys -z)”)” echo ‘CuptiUseRawGpuTimestamps=false’ >> “$(nsys -z)”
This is needed until the timestamp conversion is handled correctly in a future version.
Hello, I’ve met the same problem and I try to deal with it using
mkdir -p "$(dirname "$(nsys -z)")" echo 'CuptiUseRawGpuTimestamps=false' >> "$(nsys -z)"
however, when I check the outcome of
nsys -z
it turns out to be
/home/user/.config/NVIDIA Corporation/nsys-config.ini
there is a space in the route so simply applying the original method doesn’t work
I tried to
mkdir "NVIDIA\ Corporation" && cd NVIDIA\ Corporation/
echo 'CuptiUseRawGpuTimestamps=false' >> nsys-config.ini
but that doesn’t solve my problem.
looking for your suggestions,
Best
PhilFan
We just released Nsys 2025.5 on at Nsight Systems | NVIDIA Developer and this is one of the timing issues that was fixed in this release.
So if you update to the latest version, you won’t need the workaround.
