I’m not sure what “batching models” means in your description and why you would need to do it this way.
If you need to render model number 0, then model 1, and then both models, I assume your Trace()
function does calls to all optixTrace functions for each individual model (traversable handle) before calling all optixTrace for the next? That is, you’re not shooting the same ray into three different scene setups?
There are multiple ways to implement that with OptiX. Here are three options:
Inside the optixTrace call you define what your scene root is with the traversableHandle argument.
That defines your world space and all elements which can be reached during the acceleration structure (AS).
Means if you want to shoot rays into different scenes, using the same pipeline programs and shader binding table entries, you would just need to have different traversableHandles you can use as argument to optixTrace.
You would normally not build an instance AS and then only trace against the individual OptixInstances inside that, which wasn’t even possibly inside previous OptiX versions at all.
The fastest traversable performance on RTX boards is the IAS->GAS scene structure (means OptixPipelineCompileOptions traversableGraphFlags = OPTIX_TRAVERSABLE_GRAPH_FLAG_ALLOW_SINGLE_LEVEL_INSTANCING
Following your idea you could implement the following:
traversabeHandle[0]: Build an IAS with a single OptixInstance with identity matrix and GAS0 as child.
traversabeHandle[1]: Build an IAS with a single OptixInstance with some transform matrix and GAS0 as child.
traversableHandle[2]: Build an IAS with two OptixInstances, one with the identity matrix and GAS0, and the other with the transform matrix and GAS0.
(If your Instance 2 above actually has another transform on top, merge that into the two OptixInstance transforms.)
Each of these scenes is identified by a top-level traversable handle which has an IAS->GAS structure for optimal performance!
Now place these three traversable handles into your launch parameter block and then you can do your loop inside the ray generation program over these three traversable handles and handle each after the other.
This would work with arbitrarily many traversableHandles when you place them into a buffer and put just the pointer and size into the launch parameters which are limited to 64kB constant memory.
A less complicated scene setup would be the following using OptixVisibilityMask:
The OptixInstance structure contains a visibilityMask field which is unsigned int but is only an 8 bit value.
The optixTrace function also has an OptixVisibilityMask argument and if the AND operation of these two values is not zero, the instance is traversed. Means this works like a DIP switch with 8 switches.
So if you only build the third IAS->GAS structure containing two OptixInstance entries referencing both GAS0, and then set the visibilityMask of the instance holding GAS0 to 1 (bit 0 is set) and the other to 2 (bit 1 is set), then you can select which of three cases you want to traverse simply by setting the visibilityMask inside the optixTrace call to 1 for the untronsformed instance with GAS0, to 2 for the instance with transform over GAS0, and to 3 for both instances.
(In case you’re tracing one ray into the three different scene setups, this would be the better method.)
Building instance AS is very fast and they also do not require a lot of memory, and esp. not in your example case, so the idea with the three top-level traversable handles is probably the fastest method.
The one with the OptixVisibilityMask is more elegant.
But if I understood the idea correctly, I would first try to simply launch three times with three different traversable handles.
Example code for updating the launch parameters asynchronously here (that code is just setting the sub-frame iteration index for each optixLaunch in the benchmark mode):
https://github.com/NVIDIA/OptiX_Apps/blob/master/apps/MDL_renderer/src/Device.cpp#L1880
Related threads with caveats about not tracing against the top-level traversable handle inside a render graph:
https://forums.developer.nvidia.com/t/traverse-a-bvh-from-a-specific-node/262208
https://forums.developer.nvidia.com/t/how-to-use-optix-trace-a-child-instance/272378