Can I use Eigen library(C++ linear algebra library) in OptiX?

Hello, I am working with OptiX 7.2 on Windows Visual studio environment.

I want to perform an inverse transform of 3x3 matrix in each CUDA thread and tried to use Eigen library(C++ linear algebra library) in .cu file.

I generated a solution file using CMake after putting Eigen library folder in “…\OptiX SDK 7.2.0\SDK\cuda” and included Eigen header file in .cu file.

However, this method shows me an error “C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v10.1/include/cuda.h(53): catastrophic error: cannot open source file “stdlib.h””.

Could anyone give me advice on this situation? Any other method that is not using Eigen library is welcome.

Another NVIDIA project called Instant-ngp, uses in Eigen. You might take a look at that to see how to incorporate it in your own project.

Which CUDA compiler are you using to compile the OptiX device code? NVCC or NVRTC?

If that is using the NVIDIA CUDA Runtime Compiler (NVRTC) then this would be expected because that only compiles device code and is not aware of host compiler includes by default.

Since your working within the OptiX SDK example framework, check inside the CMake GUI if you have CUDA_NVRTC_ENABLED switched off to use the NVIDIA CUDA Compiler (NVCC) at build time only.

If NVRTC was not enabled and the stdlib.h was not found, check if the CUDA_HOST_COMPILER setting points to the correct host compiler path.

Is your task only the inversion of a 3x3 matrix?
In that case, you could also port an existing routine to an OptiX device function instead of including potentially huge library code.
There are enough examples for matrix inversion code, like here: https://github.com/nvpro-samples/nvpro_core/blob/master/nvmath/nvmath.inl#L946