NSight 4.2 + VS2010, OpenCL Trace: Error R6030 CRT not initialized [solved]

Hello,
I’m unable to get a “Performance Analysis” of my console application with NSight 4.2 (Build 4.2.0.14314) within Visual Studio 2010 Prof (SP1).
The application runs fine from command line and within Visual Studio Debugger.
When I try to start it via
Menue: Nsight->Start Performance Analysis,
Trace Application (OpenCL)
Application Control → Launch
a Dialog box appears immediatly:
Title: Microsoft Visual C++ Runtime Library
Text: Runtime Error! Program: “…”
R6030
- CRT not initialized

If I check “System” instead of “OpenCL” under Trace Settings the application runs as expected and I get a nice timeline trace.

Steps I tried so far:

  1. Deinstallation of NVidia display driver via Windows device manager (including software deletion),
    Installation of NVidia display driver 344.65 (notebook, 64bit)
    → R6030
  2. Installation of Visual Studio Redistribuables:
    VS2010 SP1 x64
    VS2010 SP1 x86
    VS2012 Update 4 x64
    VS2012 Update 4 x86
    VS2013 x64
    VS2013 x86
    → R6030

My environment:

DELL Precision 6400 (i7-2860QM)
NVidia Quadro 2000M (2 GByte)
Windows Prof 64bit SP1
NVidia display driver 344.65 (notebook, 64bit)
Visual Studio 2010 Prof SP1
CUDA 6.5.14 (notebook, 64bit)
NSight 4.2 (4.2.0.14314)

NVidia Control Panel System information:

[Anzeige]
Betriebssystem: Windows 7 Professional, 64-bit (Service Pack 1)
DirectX-Version: 11.0
GPU-Prozessor: Quadro 2000M
Treiberversion: 344.65
Direct3D-API-Version: 11
Direct3D-Funktionsebene: 11_0
CUDA-Kerne: 192
Kerntakt: 550 MHz
Shadertakt: 1100 MHz
Speicher-Datenrate: 1800 MHz
Speicherschnittstelle: 128-Bit
Speicherbandbreite: 28.80 GB/s
Gesamter verfügbarer Grafikspeicher: 9962 MB
Dedizierter Videospeicher: 2048 MB DDR3
System-Videospeicher: 0 MB
Freigegebener Systemspeicher: 7914 MB
Video-BIOS-Version: 70.06.36.00.03
IRQ: 16
Bus: PCI Express x16 Gen2
Geräte-ID: 10DE 0DDA 04A31028
Teilenummer: 1076 0503

[Komponenten]
nvui.dll 8.17.13.4465 NVIDIA User Experience Driver Component
nvxdsync.exe 8.17.13.4465 NVIDIA User Experience Driver Component
nvxdplcy.dll 8.17.13.4465 NVIDIA User Experience Driver Component
nvxdbat.dll 8.17.13.4465 NVIDIA User Experience Driver Component
nvxdapix.dll 8.17.13.4465 NVIDIA User Experience Driver Component
NVCPL.DLL 8.17.13.4465 NVIDIA User Experience Driver Component
nvCplUIR.dll 8.0.800.0 NVIDIA Control Panel
nvCplUI.exe 8.0.800.0 NVIDIA Control Panel
nvWSSR.dll 6.14.13.4465 NVIDIA Workstation Server
nvWSS.dll 6.14.13.4465 NVIDIA Workstation Server
nvViTvSR.dll 6.14.13.4465 NVIDIA Video Server
nvViTvS.dll 6.14.13.4465 NVIDIA Video Server
NVSTVIEW.EXE 7.17.13.4465 NVIDIA 3D Vision Photo Viewer
NVSTTEST.EXE 7.17.13.4465 NVIDIA 3D Vision Test Application
NVSTRES.DLL 7.17.13.4465 NVIDIA 3D Vision Module
nvDispSR.dll 6.14.13.4465 NVIDIA Display Server
NVMCTRAY.DLL 8.17.13.4465 NVIDIA Media Center Library
nvDispS.dll 6.14.13.4465 NVIDIA Display Server
NVCUDA.DLL 8.17.13.4465 NVIDIA CUDA 6.5.29 driver
nvGameSR.dll 6.14.13.4465 NVIDIA 3D Settings Server
nvGameS.dll 6.14.13.4465 NVIDIA 3D Settings Server

Except OpenCL domain, are other domains (CUDA, DX, OpenGL) workable? Is your 3D setting on NVIDIA card? (Right click on Desktop → NVIDIA Control Panel → Mnage 3D settings → Global Settings → Preferred graphics processor)

Thanks a lot for your reply.

Because of my german-language Win7 I couldn’t find the exact option in NVidia Control Panel.
I changed “3D-Einstellungen / Grundprofil / OpenGL-rendernde GPU” from “Automatische Auswahl” to “Quadro 2000M”. But this didn’t change anything. I’m still getting error R6030 if I try to trace OpenCL events.
http://www.intechno.de/share/NvControlPanel_GPU_Selection.png

I have tested other Trace Settings/Domains (one by one):

  • CUDA (Driver API + Runtime API)
  • DirectX (API Trace)
  • OpenGL (API Trace)

No errors were reported. But because there are no CUDA, DirectX or OpenGL calls in my program, no events were recorded.


In the meantime I examined the issue a little bit deeper:

  1. Point in time when error message occur: If I start my OpenCL program via NSight, the error message dialog appears before any source line in main() function has been executed. A debug message with "fprintf(stdout,"Hello!\n"); fflush(stdout);" in the first line of main() function does not show up. http://www.intechno.de/share/NSight_SettingsAndError_small.png
  2. Point in program where it stops: As a shoot in the dark I started "Process Explorer" (Sysinternals) while the error message dialog (R6030) is waiting for user input. The environment variables seemed to be correct. PATH contained "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v6.5\bin" as the very first entry.

    Other possibly interesting variables might be:

    ANALYSIS_OPENCL=1
    CUDA_PATH=C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v6.5
    CUDA_PATH_V6_5=C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v6.5
    NEXUS_LAUNCH_FLAGS=ANALYSIS_OPENCL
    NSIGHT_ANALYSIS_SESSION_OUTPUT_PATH=R:\Temp\TestApp150112_009
    NSIGHT_LAUNCHED=1
    NVTOOLSEXT_PATH=C:\Program Files\NVIDIA Corporation\NvToolsExt\
    

    At this point in time 3 threads were running:
    http://www.intechno.de/share/RunningThreads.png

    Stack traces for each of them:
    http://www.intechno.de/share/Stack_mainCRTStartup.png
    http://www.intechno.de/share/Stack_NexusInjectionDummy.png
    http://www.intechno.de/share/Stack_EtwDeliverDataBlock.png
    None of them was executing user code.

  3. A new clean Visual Studio project: A completely new "Hello World" program succeeded the OpenCL Trace (of cause without any OpenCL events recorded).

Is there any further information, that might be useful to examine the issue?
Are there any special project / compiler / linker settings required?

I don’t now why, but now it is working …

After playing around with debuggers and system monitors, I guess that there are DLL initialization conflicts. The error appears before the main() function starts.

My original program depends on (at least) three other DLLs: PThreadsW32, BinLog, XLDriverLibrary

I have created a clean VS2010 test project with OpenCL and inactive calls to these DLLs. Libraries have been linked to the executable via import libraries. Normal program flow skips all calls to the DLL. Only DLL initializations (DLLmain) should be performed during program startup before it enters the main() function.

I have tested only one DLL at a time:
pthreadVC2.dll → ok
binlog.dll → ok
vxlapi.dll → ERROR R6030 (as described above)

“dumpbin.exe /IMPORTS vxlapi.dll” shows 6 dependencies to other windows DLLs:
wmm.dll
iphlpapi.dll
ws2_32.dll
setupapi.dll
kernel32.dll
advapi32.dll

I have used “Process Monitor” (Sysinternals) to log all events while the test application was running. A lot of information got recorded. For me it’s impossible to understand all details. One little thing I have noticed:

The initialization of vxlapi.dll is making calls to setupapi.dll. The call stack shows that this leads to an implicit call to LoadLibrary(). According to VS2010 Help “DllMain entry point” that is not allowed. I don’t know, if this is actually problematic. As I explained above, the test program is running fine in a console (without NSight). But within NSight all calls to LoadLibrary() are intercepted by Nvidia.Nexus.Injection.dll. Are these interception functions prepared for this? I don’t know.

But anyway …

The vendor of vxlapi.dll provides a second way to initialize and use the DLL. Instead of linking to vxlapi.lib the user may call LoadLibrary() / FreeLibrary() to load and unload the DLL. Source code that hides the details is provided by the vendor. The DLL gets automatically loaded at first call of xlOpenDriver(). (You need to call it anyway.)

And yep, this kind of initialization avoids the R6030 error. OpenCL profiling is now working as expected.

To those who are interested in inspecting the issue further, I have linked to two VS2010 projects:
[url]http://www.intechno.de/share/NSightTest_vxlapi_auto_init.7z[/url]
[url]http://www.intechno.de/share/NSightTest_vxlapi_manual_init.7z[/url]
the first links to vxlapi.lib. This is leading to an automatic DLL initialization and the error R6030. The second project is using LoadLibrary() and is running fine.

Glad to hear that you get a workaround. We have reproduced your problem with attached project and passed it to developers. Hope we could fix it in the upcomming release. Thanks a lot for using Nsight and reporting issues.