I’m guessing that the most resilient approach would be to create a new
missprogram for a differentray typethan what I’m using for theclosesthitfunction with a single payload value for confirming a miss/hit. Would this be an addition to the SBT?
Yes, the number of ray types is indexed with the optixTrace argument SBTstride for the hit records and the explicit missSBTIndex for the miss program (this is getting important below).
https://raytracing-docs.nvidia.com/optix8/guide/index.html#shader_binding_table#sbt-trace-offset
See the SBT indexing formula here:
https://raytracing-docs.nvidia.com/optix8/guide/index.html#shader_binding_table#accelstruct-sbt
Mind that this SBTstride value is only four bits wide, so there is a maximum of 16 ray types, which is usually not a problem because it’s easily possible to let the same ray type handle different things by indicating the current use case with some payload flag. That’s effectively what you tried and I would still expect that to work.
Usually (see below!) if you previously only used one ray type and now use two, your optixTrace SBTstride value changes from 1 to 2 and you would need to provide twice as many miss and hit records inside your SBT. See the missRecordCount and hitgroupRecordCount arguments here:
https://raytracing-docs.nvidia.com/optix8/guide/index.html#shader_binding_table#layout
The additional hit records are not required when your SBT contains no anyhit programs!
I’d repurpose the miss function so that if a ray cast from the hit point toward the receiver is a miss, then the receiver is visible from the hit point.
Note that the fastest visibility ray implementation, which only needs an additional miss program inside the SBT when there are no anyhit programs on the visibillity ray), is described here:
https://forums.developer.nvidia.com/t/anyhit-program-as-shadow-ray-with-optix-7-2/181312/2
I’m using that inside my examples.
Read this comment describing the visibilty/shadowray implementation and why this does not need additional hit records inside the SBT!
https://github.com/NVIDIA/OptiX_Apps/blob/master/apps/rtigo12/src/Device.cpp#L692
https://github.com/NVIDIA/OptiX_Apps/blob/master/apps/rtigo12/shaders/miss.cu#L42
https://github.com/NVIDIA/OptiX_Apps/blob/master/apps/rtigo12/shaders/brdf_diffuse.cu#L186
As opposite example, that slower but general purpose shadow ray with anyhit programs is implemented here:
https://github.com/NVIDIA/OptiX_Apps/blob/master/apps/rtigo9/src/Device.cpp#L683
https://github.com/NVIDIA/OptiX_Apps/blob/master/apps/rtigo9/shaders/anyhit.cu#L88
https://github.com/NVIDIA/OptiX_Apps/blob/master/apps/rtigo9/shaders/closesthit.cu#L216
(Looks like something isn’t working right with these links on github right now.
They sometimes jump to the wrong line. Scroll down to the highlighted line number then.)
Note the different ray flags and SBT indices and strides inside the shadow ray optixTrace call of the two examples’ closest hit program!
Another fast method is to use the Shader Execution Reordering optixTraverse call and check that for hit or miss. That is shown inside the OptiX SDK optixPathTracer example.