Hello
Recently I’m trying to apply deep learning to OptiX, other than denoiser.
More specifically, I want to regress some values using neural network for each ray, in order to optimize path tracing.
This is not a post-processing. I have to run neural network while tracing rays.
However, I couldn’t find any resources related to it.
Is it possible to utilize existing deep learning frameworks in OptiX?
If so, can you give me any example?
Thanks.
You cannot run existing deep learning inferencing networks inside an OptiX launch because OptiX controls the work scheduling.
https://raytracing-docs.nvidia.com/optix7/guide/index.html#introduction#general-description
“… applications cannot use shared memory, synchronizations, barriers, or other SM-thread-specific programming constructs in their programs supplied to OptiX.”
But hopefully all CUDA inferencing kernels make heavy use of exactly that.
Means you either would need to implement the inferencing inside an OptiX device program or do the inferencing outside of OptiX.
A possible option would be a wavefront rendering approach, for example:
1.) Generate a number of primary rays into CUDA buffers,
2.) launch that ray wavefront using OptiX,
3.) evaluate what you hit inside OptiX,
4.) store all data per ray required by the inferencing into CUDA buffers,
5.) run your deep learning inferencing on these CUDA buffers outside of OptiX,
6.) generate continuation rays depending on the outcome,
7.) goto 2.
In OptiX 7 all data transport happens via native CUDA buffers.
Inside the OptiX SDK 7.0.0 you can find an example named optixRaycasting which shows how to do steps 1.-4.
Issues with this approach: