Well, I assumed the acceleration structure hierarchy to be working a bit like scene graphs, and so transformations would be applied to “everything that is attached” to a transformation node
No, the transform list, which is the ordered list of all transform matrices (instance, motion, static) along a path from the traversable handle used inside the optixTrace call down to the leaf GAS nodes inside the render graph are used to inverse-transform the ray into object coordinate space to do the ray-primitive intersection.
Note that the ray is in different coordinate spaces in the different program domains.
Raygen, closest hit and miss programs have it in world space. anyhit and intersection have it in object space.
The “world space” coordinate system is defined by the current transformation list.
The transformations can’t be applied automatically to your vertex attributes, because OptiX doesn’t know anything about the device code like the closest hit programs you’re providing.
So, I was wondering what the function
optixGetTransformListHandleassumes as index value (unfortunately no further description in the API documentation).
https://raytracing-docs.nvidia.com/optix7/guide/index.html#device_side_functions#transform-list
Is it
0for the transformation matrix of the first instance and1for the matrix of the second? And is there a preferred way to query this value? Right now, I added a value containing an index of the current model (either 0 for the ground or 1 for the center object) to the sbt_data.
No, there are two errors in your get_transforms() function.
You’re using the wrong transform list index and you didn’t copy the third rows of the matrices which is the reason for your NaN results.
OptixTraversableHandle handle = optixGetTransformListHandle(active_instance); // BUG
for (int i = 0; i < 2; ++i) // BUG
Maybe just use these:
https://github.com/NVIDIA/OptiX_Apps/blob/master/apps/intro_runtime/shaders/closesthit.cu#L46
or these which work with the handle:
https://github.com/NVIDIA/OptiX_Apps/blob/master/apps/rtigo10/shaders/transform.h
https://github.com/NVIDIA/OptiX_Apps/blob/master/apps/rtigo10/shaders/brdf_diffuse.cu#L78
Each object will report its current transform list from root traversable (the handle used inside the optixTrace call) to the leaf node containing the hit primitive.
In a render graph using OPTIX_TRAVERSABLE_GRAPH_FLAG_ALLOW_SINGLE_LEVEL_INSTANCING there is only one instance matrix above each GAS, and that is the index 0 in the transform list.
As you see in my introduction example code, I have multiple instances with different transforms and always use the hardcoded index 0 to get the current transformation matrix and its inverse inside the closest hit program.
If you want to see how to concatenate the transform matrices inside a deeper hierarchy (when using multiple levels of IAS or motion or static transforms) in the OptiX render graph, have a look at the OptiX helper functions provided inside the optix_7_device_impl_transformations.h header.
I’m using that inside the intro_motion_blur example here:
https://github.com/NVIDIA/OptiX_Apps/blob/master/apps/intro_motion_blur/shaders/closesthit.cu#L72
Of course that could also be used for the single instance case but would be overkill.
(If you wonder, the transform functions I use in my other examples are older than that helper header, which also implements the individual transforms for points, normals and vectors.)
Right, invalid ray exceptions normally happen when any of the ray origin, direction, t_min, t_max or time components are NaN or if t_min >= t_max or t:min < 0.0f.