Intermittent hanging in debug mode only

Gaming and Visualization Technologies Visualization OptiX
1

I’m hitting a new problem since updating to VS 2022 Version 17.10.0. I’m getting intermittent hangs when starting up my project in Debug mode only. No problems in Release mode.

The hangs happen at different places when starting up and sometimes but not always I see this message that I don’t see in Release mode.

[ 3][ MEMORY]: cblCmdListBufferSize insufficient, attempting allocation

Here’s an example.

[ 4][ DENOISER]: using cuda device “NVIDIA GeForce RTX 3090” (8.6), buffers: fp16, xmma/xmma-jit convolution, rt v12050
0.334137 DEBUG [13124 PostProcessHandler.cpp->initialize:44] Denoiser State Buffer: 8852096
0.334391 DEBUG [13124 PostProcessHandler.cpp->initialize:45] Denoiser Scratch Buffer: 165019712
[ 4][ DENOISER]: layers created for resolution 1000 1000, input inference channels 8, output channels 3
0.334723 DEBUG [13124 PostProcessHandler.cpp->initialize:46] Compute Normalizer Scratch Buffer:128
2.076901 DEBUG [13124 Util.h->~ScopedStopWatch:107]
ADDING SKYDOME took 1710 milliseconds
[ 3][ MEMORY]: cblCmdListBufferSize insufficient, attempting allocation

I get this in Cuda 12.3 and 12.5. I updated to 12.5 because that is not the cause of the compile problem with ‘__msvc_iter_core.hpp’

OptiX 8
Cuda 12.5
Geforce RTX 3090 Driver 555.85

2

Hey @Bird33,

I’d be curious to hear if you see the same thing on any SDK samples.

Also is it possible to check against CUDA 12.0? I’ve seen a few compile issues in the past with CUDA 12.5, and it is getting pretty far ahead of what OptiX 8 shipped against. Newer CUDA toolkits are supposed to be okay in general, but we do get surprised with changes from time to time, so we always recommend when running into issues that could be caused by changes in CUDA to stick with the CUDA version mentioned in the OptiX release notes, which is CUDA12.0 with OptiX 8.0.

–
David.

3

Sorry for the delay!

I’d be curious to hear if you see the same thing on any SDK samples.

Can’t compile them because of the " unsupported Microsoft Visual Studio version! " error introduced by the latest Visual Studio update. I’m not a cmake user and don’t know how to add the ‘-allow-unsupported-compiler’ flag to get them to compile.

I reverted to Cuda 12.0 for my own project but still hit the hangs. I tried compiling to .ptx files instead of optix-ir but still get them. I almost always see this message ‘cblCmdListBufferSize insufficient, attempting allocation’ when it hangs.

I beginning to think this might be a driver issue. I’m have a Geforce RTX 3090 and when I updated to Cuda 12.5 a new driver was installed (555.88) and I started getting intermittent hangs after that. I’ve since updated to version 555.99 but that did not fix anything for me…

On another note, I managed to get my forever project hooked into LightWave3d as a plugin. It never ceases to amaze me how quickly OptiX can rebuild a BVH. :)

Thanks for the help

1 Like
4

Can’t compile them because of the " unsupported Microsoft Visual Studio version! " error introduced by the latest Visual Studio update. I’m not a cmake user and don’t know how to add the ‘-allow-unsupported-compiler’ flag to get them to compile.

That should be possible to add inside the FindCUDA.cmake script inside the
macro(CUDA_WRAP_SRCS cuda_target format generated_files)

There are things like this inside the macro
set(nvcc_flags ${nvcc_flags} -m64)
and you should be able to add other NVCC options the same way:
set(nvcc_flags ${nvcc_flags} -allow-unsupported-compiler)

(I cannot test this. I haven’t upgraded and I’m not at work.)

It never ceases to amaze me how quickly OptiX can rebuild a BVH. :)

Yup, I’ve added SRT animation to my GLTF_renderer and currently add morphing and skinning which is almost complete and the OptiX AS rebuilds and esp. updates are pretty fast.

5

Thanks. Modifying FIND_CUDA.cmake worked and was able to compile the SDK samples. I tested most and did not get any hangs. But they are intermittent for me so I’ll try some more when I have more time.

I did hit a problem in the ‘optixDisplacedMicromesh’ sample in debug mode.
Caught exception: OPTIX_ERROR_INVALID_INPUT: Optix call ‘optixModuleCreate( state.context, &module_compile_options, &state.pipeline_compile_options, input, inputSize, LOG, &LOG_SIZE, &state.module )’ failed: C:\ProgramData\NVIDIA Corporation\OptiX SDK 8.0.0\SDK\optixDisplacedMicromesh.cpp:892)
Log:
COMPILE ERROR: Optimized debugging is not supported. Module is built with full debug info, but requested debug level is not “OPTIX_COMPILE_DEBUG_LEVEL_FULL”.

6

Yes, that is wrong inside the OptiX SDK 8.0.0 example code and fixed internally already.
That should be something like this instead.
(The release mode settings are all zero which is handled by the {} initialization.)

void createModule( DisplacedMicromeshState& state )
{
    OptixModuleCompileOptions module_compile_options = {};
#if !defined( NDEBUG )
    module_compile_options.optLevel   = OPTIX_COMPILE_OPTIMIZATION_LEVEL_0;
    module_compile_options.debugLevel = OPTIX_COMPILE_DEBUG_LEVEL_FULL;
#endif
...