I had few confusions and I wanted to clarify those.
In OptiX, is each ray is associated with one thread?
I’m trying to understand divergence issue in traversal and I recently came across this paper https://research.nvidia.com/sites/default/files/pubs/2009-08_Understanding-the-Efficiency/aila2009hpg_paper.pdf. It talks about persistent threads and how one fast warp wait for another slow warp in the same thread block to finish executing. And they showed that use of persistent threads could solve that problem. I was wondering if OptiX does the same? What happens to a warp which finishes executing its traversal? Does it wait for other warps in the same thread block?
Also I would appreciate if someone could point at resources where I could study working of RT cores and CUDA cores in somewhat low level for a ray tracing application. If there is any resource on low level understanding of how OptiX works, that would be great too.
In OptiX, you control when rays are cast, using optixTrace() or optixTraverse(), and you can cast multiple rays during the course of executing a single thread. Each thread will only traverse one ray at a time, but you can have multiple rays associated with a single thread.
Divergence and thread scheduling is lower level than OptiX, that’s more at the level of GPU hardware. When there’s code divergence within a warp, for example an if() { ... } block where the condition is not constant across the warp, the inactive threads will wait for the active threads to execute the block before finishing. This is fairly universal to all GPUs and the SIMD/SIMT execution model. If warps within a block diverge far from each other then yes, some of the hardware might be idle while the long-tail warps finish. The CUDA Programming Guide mentions “There is no guarantee of scheduling between thread blocks, so a thread block cannot rely on results from other thread blocks, as they may not be able to be scheduled until that thread block has completed.”
This topic is covered in the CUDA Programming Guide, a great resource for understanding CUDA cores in depth, which I recommend reviewing regularly - I review it myself often. ;)
The best recommendation is to avoid divergence when possible. You can measure it using Nsight Compute, and then research techniques for preventing both code & data divergence. This is, more or less, the most discussed topic in GPU programming, and there are many clever techniques for reducing divergence. After that, when using OptiX, the next thing to reach for is Shader Execution Reordering (SER). SER will let you reorder threads, on the fly during execution, to reduce divergence, based on which shader programs are called, and locality information about the ray, as well as any optional sort information you want to add. SER is much easier to use than persistent threads, and may solve your divergence problems. If you tell SER which warps are running longer than others (by using the coherence hint bits) then SER will group the long-running warps together, and group the short-running warps together, and this can help prevent SMs from being held up by a single warp or single thread.
Some good resources for understanding SMs and RT Cores and OptiX include: