I am trying to profile my julia code like in Profiling · CUDA.jl.
It works fine if I run
nsys profile C:\Users\ddt00\AppData\Local\Programs\julia-1.8.1\bin\julia-1.8.1.exe
However, when I was running
nsys launch C:\Users\ddt00\AppData\Local\Programs\julia-1.8.1\bin\julia-1.8.1.exe
it returns
command ignored without any error and nothing happened.
Is there any hint to fix it?
Try disabling os runtime tracing, i.e. specify --trace without osrt (e.g. nsys launch --trace=cuda,nvtx julia). There is a bug in 2023.2 with how RUNPATH interception works: I have opened a ticket and Nvidia have told me it will be fixed in the next release.
I tried it but it still returns the information command ignored without launching nsys.exe nor julia.exe.
The two most common reasons for that message appearing are:
(a) The Nsight Systems agent is not in a state where it is ready to start a new application trace (e.g. it is already tracing another application, it is processing / cleaning up a previous trace, …). It might be accidentally stuck in such a state from a previous trace.
(b) There is some sort of permission problem with the launch such as the executable not being able to spawn child processes, the target executable not being reachable, etc.
Things to try:
(1) Close all instances of nsys.exe still running in the background (one or more of them may be stuck in the background), and the GUI client nsys-ui.exe if it is running then try again.
(2) Under your %TEMP% directory locate a directory named nvidia, which should have a subfolder named nsight_systems. Delete the subfolders agent_files and injection_files if they are present as well as any files named *.lock directly under the nsight_systems folder.
(3) Try running the command as an administrator.
Hope one of these helps!
I tried this one and it returns issues as follows:
PS C:\Users\ddt00> nsys launch --trace=cuda,nvtx C:\Users\ddt00\AppData\Local\Programs\julia-1.8.1\bin\julia-1.8.1.exe
Agent daemon did not report success until timeout
PS C:\Users\ddt00> nsys launch --trace=cuda,nvtx C:\Users\ddt00\AppData\Local\Programs\julia-1.8.1\bin\julia-1.8.1.exe
Dynamic exception type: class std::invalid_argument
std::exception::what: invalid stoi argument
I found another ticket with the same issue, and the solution is to use “profile” instead of “launch”. However, is it possible to fix the issue with “launch” since I want to profile my code in an interactive mode？
I have opened a ticket in our bugs system to track this issue, we will try to reproduce it and see what exactly is the problem.
If nsys profile works for you and you want to manually start the capture, you can, as a workaround, you can set --delay=100000 or -y 10000 (or some other value in seconds that is much higher than the delay you want). Then, you can call nsys start any time before the delay expires and the capture starts automatically to emulate the behavior of the nsys launch command.
Thanks again for the report!