"illegal memory access was encountered" in CUDAOutputBuffer, gets triggered by multiple materials

The complete error message is the following:

CUDAOutputBuffer destructor caught exception: CUDA call (cudaGraphicsUnregisterResource( m_cuda_gfx_resource ) ) failed with error: 'an illegal memory access was encountered' (C:\ProgramData\NVIDIA Corporation\OptiX RayTracer Project\SDK\sutil/CUDAOutputBuffer.h:147)

Caught exception: CUDA error on synchronize with error 'an illegal memory access was encountered' (C:\ProgramData\NVIDIA Corporation\OptiX RayTracer Project\SDK\optixBasicRayTracer\optixBasicRayTracer.cpp:121)

CUDAOutputBuffer destructor caught exception: CUDA call (cudaFree( reinterpret_cast<void*>( m_device_pixels ) ) ) failed with error: 'an illegal memory access was encountered' (C:\ProgramData\NVIDIA Corporation\OptiX RayTracer Project\SDK\sutil/CUDAOutputBuffer.h:139)

I’m trying to create an OptiX 7 version of the Ray Tracing in One Weekend book, which has 3 different materials in it. All 3 materials are implemented, and work individually, but when I try to mix them in the scene, by having multiple SBT records in the program for the different materials, the error above is what I get.

It happens as soon as I try to assign an SBT index value of over 0 in the IndexOffsetBuffer. If every object has an SBT index of 0, the code runs fine.

I have an RTX 3090, so the issue is definitely not about running out of VRAM. How should I go about debugging this?

When you say you’re using the buildInput sbtIndexOffsetBuffer that means your scene primitives are all in a single geometry acceleration structure (GAS)?
If yes, are all indices inside the sbtIndexOffsetBuffer in the range [0, numSbtRecords - 1]?
That is, did you set the proper buildInput.numSbtRecords for that GAS?

Please read this chapter inside the OptiX Programming Guide:

The most crucial part of that is the formula for the sbt-index calculation.
The example in chapter 7.3.2 describes how to use multiple SBT records per GAS.

Other than that, please have a look at this recent post explaining how to find illegal access errors:

Hi, thanks for the answer!

Enabling validation mode has given me a more concrete error code, which is

[ 2][       ERROR]: Error recording event to prevent concurrent launches on the same OptixPipeline (CUDA error string: an illegal memory access was encountered, CUDA error code: 700)
Error recording resource event on user stream (CUDA error string: an illegal memory access was encountered, CUDA error code: 700)

Unfortunately I’m still not sure what is causing it. I tried to look for things that might be causing concurrent launches, but I’m not aware of anything in the code that would do that. Although maybe I’m just not quite getting what the error message really means.

Another thing that I remembered is that when I tired to add an accumulation buffer to my Params, it also resulted in an identical “illegal memory access” error, but I can’t really figure out the problem from that angle either. The best option would probably be to start over from a simple project at this point, and build it up again, piece by piece, but unfortunately I don’t have the time to start it over, so I’m not sure what road to continue on?

If you’re not actually doing multiple concurrent launches, then that error message is a red herring because that is then just the first time the illegal access during your asynchronous optixLaunch kernel could be caught.
It’s happening earlier with validation enabled because that adds synchronizations to make errors visible nearer to the API call where they occured.

It’s not really possible to say what went wrong inside your code with the given information so far.
But if everything is working with only 0 indices inside the sbtIndexOffsetBuffer and crashes with any value greater than 0, then you’re most likely setting up your shader binding table incorrectly or call the optixTrace call with the wrong parameters.
That shouldn’t be too difficult to find but I would need to see a lot more of your code to be able to say what is going wrong exactly.

You could try running Nsight Visual Studio Edition with all OptiX device programs compiled as *.optixir and with debug information and OptixModuleCompileOptions set to no optimizations and full debug level.
See https://raytracing-docs.nvidia.com/optix7/guide/index.html#program_pipeline_creation#program-input
That might be able to detect where the illegal access happens inside your device code.
See this post for what is working and what not with debugging: