In OptiX 7.3 (and hopefully all subsequent minor versions), is that possible that, given an array of OptixInstance’s, the OptixInstance::traversableHandle of some of them can be set to null to indicate an unavailable child AS and optixAccelBuild automatically ignores those?
I have tried this on a few SDK samples that use IAS and all of them work as expected without errors. Strange enough I also tried value of 1 and it works without problem; any other random values produce a pretty consistent and expected crash.
I looked up from programming guide and the closest statement found is about optixTrace in device code which can take a null handle so it only invokes the miss shader, but nothing related to IAS build. I wonder if that’s intended or just a result of undefined behaviour?
Yes this is an intentional feature of OptiX that you can set your traversable handle to zero and it will be ignored. Having the 1 value work in place of 0 is undefined behavior, that might not always be the case, and any non-zero value that is not a valid traversable handle is invalid input whether it crashes or not.
The goal behind this feature is to allow hiding and showing of instances when using a BVH update operation (OPTIX_BUILD_OPERATION_UPDATE). So the intended use would be to have all valid & non-zero traversable handles during the initial BVH build operation, and then you can set some to zero and update the BVH when you want to hide them. BVH updates are typically much faster than builds, but updates require that the number of instances remains constant after the initial build.
You want to avoid passing zero for traversable handles during an initial build operation, since that will lead to the empty bounding box being placed somewhere arbitrary in your acceleration structure. If you then pass non-zero during an update operation, you can end up with a wildly non-optimal AS, and tracing might be very slow.
You’re right that this isn’t well documented enough, so we will take the note to improve the documentation in the near future, thanks for pointing it out.