I have a rendering engine written in Vulkan using Nvidia ray tracing extensions. As a reference I’ve build in OptiX 6.0 denoiser and I’ve encounter a peculiar problem. If and only if, I’m using RTX with OptiX (specified using RT_GLOBAL_ATTRIBUTE_ENABLE_RTX flag), then I get an error, which depends on Vulkan/OptiX initialization order.
If I create OptiX context before creating device in Vulkan (vkCreateDevice). Then vkCreateDevice will fail with error code: VK_ERROR_EXTENSION_NOT_PRESENT.
If I create Vulkan device before OptiX context, then OptiX throws the following error while calling OptixContext->createBuffer(…):
“Unknown error (Details: Function “_rtBufferCreate” caught exception: Encountered a rtcore error: m_exports->rtcInit( debugLogLevel, debugLogCb, debugKnobs ) returned (11): Library was already initialized with different parameters)”
I have 2080Ti and Titan Xp in my PC and I made sure everything is forced to run on 2080Ti, Vulkan and CUDA. For debugging purposes I also tried all GPU device/context creation combinations including multi-gpu configurations (for example CUDA on Titan Xp, Vulkan on 2080Ti), but nothing works. Keep in mind that the code works fine if OptiX is running with RTX off, but the denoising is quite slower, the goal is to run it with RTX on - that’s when the errors popup.
Also note that OptiX denoising runs fine with RTX on if it’s run in a separate application, problems only arise when combining it with Vulkan ray tracing in the same application.
I’m on Windows 10 with latest updates and drivers, using CUDA 9.0.
Any ideas how to get around this?