Optix to Direct3d 11

Hi developers! We are working on a real-time render engine on D3D11 and we want real-time reflections that are not so limited like the rasterized type of reflcetion techniques (SSR, Planar, etc). But we are struggling at outputting the Optix result (only reflection pass) to Direct3d… We’ve looked all through the documentation and guides but still we were not able to find anything releated to Direct3d interop. :( we were hoping to find some help here.


I would recommend using OptiX 7 for that.
This is more involved with earlier OptiX versions which do not have native support for DX interop, but there have been solutions posted in this sub-forum. You’ll find them when searching for “DX11” in the search field on the top right while being in the OptiX sub-forum topics view.

I haven’t done that with DX11 myself since I’m only using OpenGL and Vulkan, but the basic mechanism should be the same because this is all about native CUDA graphics interoperability functions.

Since all device memory handling happens with native CUDA host code in OptiX 7 versions, that is your best option.
You will only need to get that CUDA graphics interoperability working with your graphics API of choice and care about the proper synchronization.

You also need to pick the CUDA API. There are CUDA Driver and Runtime APIs.
(I prefer the CUDA Driver API due to the more explicit CUDA context handling which comes in handy with multi-GPU.)

Have a look through these links:

Many of the OptiX SDK and other example share an OpenGL pixel buffer object (PBO) to render to in OptiX before uploading it to the OpenGL texture object image for final display with a textured rectangle.

Have a look through this file of one of my OptiX 7 examples and search for the different CUDA OpenGL interop cases inside the m_interop clauses.
Except for the graphics API specific resource management, the remaining CUDA interop calls are graphics API agnostic.

You probably need the other direction as well when using a G-buffer from your raster engine to be able to determine from where to shoot rays for the reflections.

Note that DX12 DXR and Vulkan Raytracing would be the other APIs with which you could do raytracing, and that would be much more efficient than to bolt something onto a DX11 engine from a separate API, because the resources in the rasterizer and raytracer would simply be shared when using DXR or Vulkan RT.

Hi droettger, thank you for the quick answer!

Yes, we’re using Optix 7. Thank you for the links, I will check these out… And that’s what I thought, maybe I should look at the OpenGL interop and just translate it into DX. We’re actually using NVidia’s VXGI and the DX12 version is buggy as hell! So we are stuck with the DX11 version which worked perfectly, it’s cheaper than a ray-traced GI solution and easily supported on old GPU’s but well… Anything’s perfect so reflections from a voxel approach are quite bad looking so we thought it could be complemented with Optix just for the reflection part… Now I’m starting to think we will have to drop the VXGI entirely and find another solution or just make this work somehow haha! Ah god! Anyways… Thanks a lot for the help!!! :D

(EDIT: Oops, I misread your previous reply and thought you wanted to add a another interoperability indirection.)
Yes, look at the CUDA-OpenGL interop examples and port the API specific CUDA calls to DX11.
If more help is needed one of my colleagues has implemented DX12 interop with CUDA/OptiX 7, but that should work just the same from CUDA API side.

We’re actually using NVidia’s VXGI and the DX12 version is buggy as hell!

Did you file bug reports?

Again, please have a look at these DX11 search results on the OptiX sub-forum:

While the threads in that only addresses issues with the old OptiX API before version 7.0.0, the part dealing with the CUDA-DX11 interop calls themselves hasn’t changed.
There is a least one code repository link where this is used, and I would expect it to become only simpler with OptiX 7 because all the device memory allocations are native CUDA host code.
CUDA graphics interop doesn’t get simpler than that.