I’m trying to run Nsight Systems 2022.1.1 on a x64 Windows app running on Server 2019. The app uses libtorch with cuda and a Tesla v100 gpu.
I’m launching it from Visual Studio 2019 integration. I’m using the default settings except for turning on cuda trace, shown below (removed sreenshot due to “new user” limitation of one graphic per topic), as well as the error message.
Sorry for the delay in responding.
Usually that message indicates that the application ended without data collection (before the profiler got started. Were you launching the application through a script? Do you need to set to trace the entire process tree?
It’s a gui application – will display a main window and wait for user input. I never see the main window.
@liuyis can you take a look at this?
Hi @scottw, could you share the project setting page? Espcially the application command line and what profiling options you enabled.
If you cannot upload more screenshot due to any limitation, you can also send to my email liuyis@nvidia.com.
Hi @scottw, I mean the project setting page for Nsight Systems. The screenshot was from Nsight Compute.
Here is an example (but please include all options you selected):
Sorry, I lost track because it had been a while. Interestingly, Nsight Compute worked fine for this app.
Does it work if you unselect Hotkey Start/Stop?
No, same error.
Could you then also uncheck “start profiling manually” - that should result in a report though the app may still crash. But we can check the stdout/err from the report for the app.
Could you upload the report so I can take a look?
Report 6.nsys-rep (484.6 KB)
Report 6-Merged-20220323-201759.etl (16 MB)
Somehow the report didn’t catch the std outputs. We can try another method, using Nsys CLI to start the profiling session to see if it works.
To do this firstly you need to find where Nsight Systems was installed on your system. After that, use the following command (please use a terminal that has Administrative privilege):
cd <your working dir>
<Nsys install folder path>/target-windows-x64/nsys profile -t none --capture-range=hotkey <your app>
(Note that this command line should provide the same setting as shown in your project)
@scottw BTW, we just had a new public release yesterday (2022.2). Could you try it just in case it’s something already fixed. It can be downloaded from https://developer.nvidia.com/gameworksdownload#?dn=nsight-systems-2022-2.
Just checking. My app doesn’t have any std output during startup.
Sorry but I’ll need to wrap up today soon. In case the CLI and 2022.2 both don’t work - is it possible to share a reproducer for me to debug at my side? (If that’s possible, just share it and I’ll look into it tomorrow)
Here is the report generated via running nsys from the command line. I still never saw the main window of my app.
report1.nsys-rep (218.7 KB)
2022.2 didn’t seem to work any better.
This is a sprawling ML application, dependent on many other installed components so it’s not easy to get you a copy.