Greetings!
I’m having problems running the NVidia NSight Systems (v2021.1.1) on a C++, OpenGL, QT application if choosing to collect the OpenGL trace while runing NSight Systems capture.
DESCRIPTION:
What happens is that the target application terminates itself before entering the ‘main()’ function. And it only happens if “Collect OpenGL trace” is checked in the capture options, otherwise it works ok. A more visually descriptive illustration can be seen from attached images where i took snapshots of a timeline for the collected report and also its Diagnostics Summary. Images are called “Big_picture_timeline.png”, “Close_up_timeline.png” and “Diagnostics_summary.png”.
FACTS:
Here are some well tested facts i can confirm in regards to the situation:
FACT_1: Target and Host are the same PC (localhost capture).
FACT_2: The application itself - without tracing - runs normally.
FACT_3: It is a graphical application running on Windows, instantiating multiple windows in runtime of which all are sharing the same OpenGL context.
FACT_4: The application is terminated before entering the ‘main()’ function of the targeted project.
FACT_5: The tracing works well with the following capture options (can be set all at once or individually):
– Sample target process
– Collect CPU context switch trace
– Collect CUDA trace
– Collect NVTX trace
– Collect WDDM trace
FACT_6: If “Collect OpenGL trace” is included in the capture options, the problem occurs.
FACT_7: The problem occurs even if “Collect GPU workload trace” and tracing all of the OpenGL functions is disabled under checked “Collect OpenGL trace” option (everything possible under this OpenGL option).
FACT_8: The problem occurs also if “Collect OpenGL trace” is the only option selected.
FACT_9: I ran all the necessary programs with administrator privilages.
FACT_10: The application is a QT 5.12.6 application.
FACT_11: Another few - much simpler - applications work well, using the same OpenGL libraries as this project.
FACT_12: The problem occurs even if all OpenGL calls are excluded from runtime execution (commented out).
FACT_13: The OpenGL version used is 4.6.
FACT_14: Since the first occurance of this problem, i have updated the GPU drivers and the result was still the same.
TARGET_TERMINATION_BEFORE_MAIN:
I know that the application terminates itself before entering the ‘main()’ function. I confirmed it by storing a dummy .txt file to desktop (tested on a normal application execution and it works) but the file wasn’t there when tracing OpenGL. This was the first thing that the main() function did and it didn’t do it.
RUNNING_FROM_CLI:
I tried running the trace capture from command line interface as well (using GIT Bash). I got the message “The target application returned non-zero exit code -1073741819” and - if i’m not mistaken - this translates to C0000005 which - on MSDN - means ACCESS DENIED. In order to run the NSight Systems i used the following command: “nsys profile -t opengl --duration 10 -o ./_OUTPUT/report_1 --force-overwrite true -w true --kill true ./MY_APP.exe”. I also tried using a much simpler command, stating only the necessary “-t opengl” but the result was the same. A full CLI output is visible from the attached image called “CLI_run.png”.
DIFFERENCE_BETWEEN_CUDA_VERSUS_OPENGL_CAPTURE_AND_QT_SUSPICION:
Then - wondering about the difference between successfully running CUDA trace and unsuccessfully running OpenGL trace - i tried to figgure the difference between the two. What i did was using “strace” command before the upper mentioned CLI call to ‘nsys’. This allowed me to see a bunch of .dll loading calls and after collecting them alphabetically and comparing them in text comparing program, i noticed, that when running OpenGL trace, two .dll files were not mentioned anywhere in the ‘strace’ output whereas they were present with the successful CUDA trace. These .dll files were called “qwindows.dll” and “qwindowsvistastyle.dll”. These belong to QT as far as my understanding goes and this is why i’m suspicious that QT has something to do with it. Combining this information with upper mentioned ACCESS_DENIED - if that really was the case - i’m thinking that maybe these .dlls weren’t reachable for some reason if OpenGL trace was used (they reside in their own folders for example ‘…exe_path/platforms/qwindows.dll’ as the QT convention dictates. It’s also worth mentioning that this was the only difference between working CUDA and non-working OpenGL trace outputs in regards to using ‘strace’ and observing .dll filenames.
CHANGING_NSIGHT_MONITOR_SETTINGS:
I also tried tempering with the NSight Monitor settings. I disabled WDDM TDR (which was enabled and set to 2 seconds), disabled “Driver Instrumentation Enabled” setting but the problem remained.
Unfortunately i cannot send you this project, it belongs to the company i work for. All i can say is that it’s fairly complex and list you the libraries that we’re using in case NSight Systems could clash with one of them. And all the smaller scale projects i’ve used to test this are working fine with OpenGL trace enabled. It’s also worth mentioning, that i couldn’t get Visual Studio’s instrumentation profiler to work with this same project (but that might have been an issue related to something else). I can also say that using Apitrace program worked with capturing the data but NSight seems the way i’d really like to go.
If you have any ideas on what i could try further or perhaps any known issues i’ve missed checking the forums, i’d be really greatful. Getting NSight to run would benefit us greatly.
I would also like to take this opportunity to compliment you on the great work you’re doing. NSight seems like a really great set of tools. Keep up the good work!
Thank you for your time and Best Regards!
SYSTEM_INFORMATION:
CPU: Intel(R) Core™ i7-6700K CPU @ 4.00Ghz
RAM: 64 GB, GDDR5
GPU: NVidia GeForce GTX 980 Ti
Driver version: 461.72
OpenGL version: 4.6
OS: Windows 10 Pro (x64)
NSight Systems: 2021.1.1
NSight (VSE): 2020.3.1.21012 (Standard)
Visual Studio: VS Professional 2019, version 16.9.0