[OptiX 7.5] Payload type mismatch errors when using OptiX-IR


I’m migrating my OptiX wrapper to OptiX 7.5.0 and particularly in implementing OptiX-IR support.

I have found that some programs that have worked properly until now output some error messages from their kernels regarding payload type mismatch when compiling a module from OptiX-IR with OPTIX_COMPILE_OPTIMIZATION_LEVEL_0 and OPTIX_COMPILE_DEBUG_LEVEL_FULL.
The error messages are like:
( 24, 48, 0) error: payload type mismatch between program and optixTrace call

However, these programs don’t use payload annotation feature (specify a non-zero value of payload size in the pipeline options and kernels don’t call things like optixSetPayloadTypes()). If I use a ptx as the input for the module as until now, the error messages disappear.

These programs still work as expected even with OptiX-IR in Release-oriented build (like, using OPTIX_COMPILE_OPTIMIZATION_DEFAULT and so on).

I have questions:

  • Is the optixTrace without the OptixPayloadTypeID parameter equivalent to the optixTrace with that parameter with payloadTypeID = OPTIX_PAYLOAD_TYPE_DEFAULT?
  • Is payload annotation still optional feature for programs using OptiX?
  • Are error messages false due to OptiX-IR? Or does OptiX-IR report some potential errors that weren’t reported with ptx.

In case you can try to reproduce the issue using my program:

  • Use optix_750 branch
  • Build the solution for VS2022 using CMake
  • Run the project “10.denoiser” with Debug build
  • optix_kernels.cu is configured to be compiled to .optixir file by using --optix-ir -G options
  • If it uses .ptx as the input file for the module, the program doesn’t output the error messages and finishes as expected.
    (Change USE_OPTIX_IR at the line 49 in denoiser_main.cpp to 0 and edit custom build tool for optix_kernels.cu to generate optix_kernels.ptx.)


My Environment:
OptiX 7.5.0 / CUDA 11.7
Driver: 516.40
Visual Studio 2022 17.2.4
Windows 10 21H2
RTX 3080 10GB

Could you check this?


Hi @shocker.0x15,

Payload annotation is still optional, and OPTIX_PAYLOAD_TYPE_DEFAULT is the default when not passing an ID. The errors could be a bug with OptiX-IR, we willl investigate. I ran into a bit of trouble with your repro, cmake complained about the assimp and googletest folders. Does it require VS2022 and/or a recent version of cmake? I currently have cmake 3.19 and VS 2019. Is this payload error something you’ve seen with any of the OptiX SDK samples?


I was using CMake 3.21.4 and VS 2022. I tried CMake 3.19.8 and VS 2019 (16.11.16) as well and I could reproduce this issue. Did you use --recursive option when cloning? (Sorry I should have mentioned this. And you need to checkout optix_750 branch.)

I tried every SDK sample which doesn’t require command line options. I couldn’t see something related to this issue.
(I’m using CMake 3.21.4 and VS 2022 for this.)

BTW, not related to this issue though,

  • optixCallablePrograms sample cannot launch the program properly with Debug build on my environment since it shows some multiple definition errors like the following which should have been fixed with 516 drivers:
    Error: Symbol '_Z3dotRK6float3S1_' was defined multiple times. First seen in: '__intersection__sphere'
  • optixSimpleMotionBlur and optixVolumeViewer samples crash with Debug build due to CUDAOutputBuffer destructor caught exception: CUDA call (cudaGraphicsUnregisterResource( m_cuda_gfx_resource ) ) failed with error: 'an illegal memory access was encountered' (C:\Users\shocker_0x15\repos\OptiX SDK 7.5.0\SDK\sutil/CUDAOutputBuffer.h:147)


Thanks! I’ll try again shortly. And thanks for the error reports in Debug configs, maybe our tests aren’t catching that config right now, but we’ll get those fixed.


Oh sorry I should have thought to use clone --recursive… that is becoming the default these days. ;) Okay, so I was able to reproduce your payload type error (and also reproduce the Debug config sample errors). I’ve filed bug reports on both.

With the payload error, just to make sure I’m seeing what you’re seeing…

The module compile options are indeed set to 0 payload types, and otherwise default values. But I also see some pipeline compile options with numPayloadValues set to 8. Could that be the issue? The pipeline options are set before the module options. I haven’t debugged very far yet, I just wanted to double-check I’m looking at the right things.


numPayloadValues in OptixPipelineCompileOptions and numPayloadTypes in OptixModuleCompileOptions are different, right?

In my understanding,

  • numPayloadValues in OptixPipelineCompileOptions is the maximum number of 32-bit payload values passed to optixTrace used in the pipeline.
  • numPayloadTypes in OptixModuleCompileOptions is the number of sets of payload semantics, each set of payload semantics is associated to optixTrace and other related programs launched during the traversal.
  • numPayloadValues must be set to an appropriate value when we don’t use payload annotation feature (numPayloadTypes == 0. This is traditional OptiX usage), otherwise it must be set to 0.

In the repro, it does not use payload annotation (i.e. numPayloadTypes == 0), so I need to set numPayloadValues to a non-zero appropriate value.

Yes, your understanding sounds correct, I just wanted to make sure I was oriented correctly. I hope we can give you an update this week. Are you setting up to switch between OptiX-IR and PTX conditionally on debugging or user preferences?



How is this going?

BTW, can we use -G instead of -lineinfo for OptiX-IR?
The release note says that we need to be sure to use -G and -O0 for debugging symbols. Is there a way to specify -O0 from the Visual Studio property window for .cu file? (Do I need to write by hand in Command Line?)

Hey, sorry I forgot to update this. The bug is still being worked on since it’s fairly deep in the system.

Yes you can use -G and -O0 for debugging, I believe -G overrides -lineinfo. In general where you add -G depends on your build system. The OptiX samples are setup to supply -G to nvcc when using the “Debug” configuration in Visual Studio. There is a little bit of cmake logic combined with some ifdefs against the NDEBUG definition. I would assume you could setup your build to send any properties from VS to your build, but I don’t actually know how to do it. If anyone else knows, please chime in.



I tried the most proper way to use OptiX-IR for my program: -G -O0 -optix-ir, but it still outputs the same error message when using OPTIX_COMPILE_OPTIMIZATION_LEVEL_0 and OPTIX_COMPILE_DEBUG_LEVEL_FULL.


Was this issue fixed?
I have updated the driver to 526.47 (and Windoes to 11 22H2), now I don’t see the issue.

My Environment:
OptiX 7.5.0 / CUDA 11.7
Driver: 526.47
Visual Studio 2022 17.3.6
Windows 11 22H2
RTX 3080 10GB

It’s possible, but we don’t know for sure yet. This may have been fixed during a switchover to a new internal compiler module, but the bug report I filed with your reproducer is still open. It is good to know that you’re not seeing the issue anymore, it may help confirm the fix faster, thanks for reporting that.


ok, I’ll see how this goes and keep this issue opened for a while.