Optix7 SDK Application crashes when clicking the ShowDesktop icon

I have read the programming guide of Optix 7. Although still confused by the new APIs, I can hardly wait to try the new SDK examples.
However, different from the legacy version, the new optixWhitted application catches exceptions when I click the show desktop icon to hide the GLFWwindow. The GLFWwindow can not be minimized as well.

Here is the message

Caught exception: CUDA call (cudaGraphicsGLRegisterBuffer( &m_cuda_gfx_resource, m_pbo, cudaGraphicsMapFlagsWriteDiscard ) ) 
failed with error: 'out of memory' (C:\ProgramData\NVIDIA Corporation\OptiX SDK 7.0.0\SDK\sutil/CUDAOutputBuffer.h:184)

Applications with imGui such as optixPathTracer, suffer from the same problem. The bug do not affect performance of rayTracing, but sometimes really annoys developers.

Hi,

I have met this problem before, too. I think the reason is that the width and height are both zero when the application window is minimized, and thus something related to OpenGL is error (in CUDAOutputBuffer) and the application crashes.

What I have done is to limit the width and height of application window above one, it is simple but works.

Right, OpenGL interop buffers must not be zero size. Looks like that is not prevented in the CUDAOutputBuffer helper class.

I filed a bug report to fix the examples. Until that is available you should be able to prevent that error by enforcing a minimum width and height of 1 in the window resize handler like yetsun recommended.

I had the same issue and using the option --no-gl-interop (so setting output_buffer_type = sutil::CUDAOutputBufferType::CUDA_DEVICE; instead of output_buffer_type = sutil::CUDAOutputBufferType::GL_INTEROP; ) worked for me.

Any news with this?

I am also getting consistent crashes with most of the SDK examples.

Caught exception: CUDA call (cudaGraphicsGLRegisterBuffer( &m_cuda_gfx_resource, m_pbo, cudaGraphicsMapFlagsWriteDiscard ) ) failed with error: ‘unknown error’ (/tmp/optix/SDK/sutil/CUDAOutputBuffer.h:196)

According to our bug database, this should have been fixed over a year ago.
Which OptiX SDK version are you using under which OS, GPUs, display driver?
(Looks like Linux from the slashes in your path.)

If you have errors with cudaGraphicsGLRegisterBuffer that indicates a problem with CUDA-OpenGL interop.
Usually that happens when the OpenGL implementation the application uses is not NVIDIA’s which would prohibit CUDA-OpenGL interop. This is either a system configuration or display driver installation issue.
Maybe check with glGetString(GL_VENDOR) inside one of the apps which OpenGL implementation is running.

If it’s actually still because the CUDA buffer for the OpenGL interop drops to zero size, then the posts above contains the necessary workaround you can apply locally, but looking into the OptiX SDK 7.4.0 sources shows the fix preventing this:

template <typename PIXEL_FORMAT>
void CUDAOutputBuffer<PIXEL_FORMAT>::resize( int32_t width, int32_t height )
{
    // Output dimensions must be at least 1 in both x and y to avoid an error
    // with cudaMalloc.
    ensureMinimumSize( width, height );
...